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For  several  reasons,  the  publication  of  this  report  was  postponed  for  several  years. 
Its  contents  were  summarized  at  the  time  of  this  author’s  oral  presentation  at  DTRC  in 
July  1985,  reflecting  the  state-of-the-art  up  to  1984.  The  manuscript  of  the  written 
version  was  completed  in  August  1987  and  is  now  to  be  published  five  years  later  in 
1992.  During  these  years  the  state-of-the-art  has  made  considerable  progress,  and  this 
author  finds  the  “review”  to  be  insufficient,  especially  in  Part  in  because  of  the  recent 
works  of  several  researchers.  In  order  to  update  the  report  this  author  has  added  a 
“Supplement  of  References,”  (Refs.  101-124)  hsting  publications  that  have  appeared 
since  1984,  together  with  some  other  pubhcations  that  he  had  failed  to  refer  to  in  the 
original  manuscript. 
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PREFACE 


The  David  W.  Taylor  Lectures  were  initiated  as  a  living  memorial  to  our  founder  in 
recognition  of  bis  many  contributions  to  naval  architecture  and  naval  hydrodynamics. 
Admiral  Taylor  was  a  pioneer  in  the  use  of  hydrodynamic  theory  and  mathematics  for  the 
solution  of  naval  problems.  The  system  of  mathematical  lines  developed  by  Taylor  was 
used  to  develop  many  ship  designs  for  the  Navy  long  before  the  computer  was  invented. 
He  founded  and  directed  the  Experimental  Model  Basin;  perhaps  most  important  of  all, 
be  established  a  tradition  of  applied  scientific  research  at  the  “Model  Basin”  which  has 
been  carefully  nurtured  through  the  decades  and  which  we  treasure  and  protect  today.  In 
the  spirit  of  this  tradition,  we  invite  an  eminent  scientist  in  a  field  closely  related  to  the 
Center’s  work  to  spend  a  few  weeks  with  us,  to  consult  with  and  advise  our  working  staff, 
and  to  give  a  series  of  lectures  on  subjects  of  current  interest. 

Our  tenth  lecturer  in  this  series  is  Dr.  Yasufiimi  Yamanouchi  who  is  currently 
Technical  Advisor  to  Mitsui  Engineering  and  Shipbuilding  Co.,  Ltd.  Dr.  Yamanouchi 
graduated  from  the  Tokyo  University  in  1943  and  received  his  Dr.  Eng.  from  there  in 
1962.  For  many  years  (1946-73),  he  served  with  the  Governmental  Research  Institute  on 
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Ship  Technology,  that  changed  its  name  from  Railway  Technical  Research  Institute 
(1946-50),  then  to  Transportation  Technical  Research  Institute  (1950-63),  and  finally  to 
Ship  Research  Institute,  Ministry  of  Transport.  He  was  Head  of  the  Ship  Performance 


Division  (1962-63)  and  Head  of  the  Ship  Dynamics  Division  (1963-69).  then  Deput>' 
General  Director  (1969-72)  and  finally  Director-General  of  the  Ship  Research  Instimte 
(1972-73),  Ministry  of  Transport  in  Tokyo,  Japan.  He  joined  Mitsui  Engmeering  and 
Shipbuilding  Co.,  Ltd.  as  Senior  Deputy  Director,  advancing  to  Technical  Director  upon 
his  retirement  in  1983.  He  contributed  to  the  construction  and  organization  of  a  new 
research  laboratory,  and  was  the  first  General  Manager  of  the  Akishima  Laboratory  of 
Mitsui  from  1975  to  his  retirement  in  1983.  He  was  Professor  at  the  College  of  Science 
and  Technology  (1982-89)  of  Nihon  University'  in  Tokyo,  Japan. 

Dr.  Yamanouchi  has  had  a  long  and  distinguished  scientific  career,  w'ith  pioneering 
publications  on  ship  dynamics,  ocean  waves  and  stochastic  processes.  He  is  no  stranger 
to  the  United  States,  having  done  research  at  the  Davidson  Laboratory  of  Stevens  Insti¬ 
tute  of  Technology  from  1957-60  and  having  served  as  Visiting  Professor  at  the  U.S. 
Naval  Academy  during  1984—85.  We  are  most  honored  that  he  agreed  to  be  a  David  W. 
Taylor  Lecturer. 


ABSTRACT 


In  Part  I,  the  general  procedures  of  the  conventional  method  of  correlation  or 
spectrum  analysis  of  a  random  process  (nonparametric  method)  are  reviewed,  stressmg 
the  statistical  reliability  of  the  results.  A  few  suggestions  for  improvmg  the  coherencies 
are  given.  In  Part  II,  the  characteristics  of  AR.  MA.  and  ARMA  models  are  discussed. 
The  model-fitting  technique  supported  by  AIC  criteria  is  introduced,  with  the  examples 
of  application  to  the  seakeeping  data.  In  Part  in.  the  statistical  treatments  of  nonlmearities 
in  random  process  analysis  are  summarized  and  reviewed.  Conclusions  are  given,  and 
future  work  is  proposed. 
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CHAPTER  1 


INTRODUCTION 


1.1  PROLOGUE 

In  the  summer  of  1951,  a  group  of  research  naval  architects  who  were  scheduled  to 
be  on  board  the  cargo  ship  Nissei-Maru  cruising  the  Pacific  from  Japan  to  the  United 
States  was  discussing  the  design  of  a  system  for  measuring,  recording,  and  analyzmg  ac¬ 
tual  ship  performance  data.  Such  a  system  was  needed  for  the  first  large-scale 
cooperative  test,'  organized  by  the  Japan  Towing  Tanl:  Committee  (JTTC)  of  the  Societj 
of  Naval  Architects  of  Japan  (SNAJ)  and  scheduled  to  start  at  tlie  end  of  the  year,  to  pro¬ 
vide  a  basis  for  the  post-World  War  II  study  of  hydrodynamic  ship  performance  in  all  the 
laboratories  and  universities  in  Japan. 

To  fmd  the  relationships  between  ship’s  propeller  revolutions,  shaft  horsepower, 
modes  of  motion,  rudder  angle,  and  so  on,  we  decided  to  record,  simultaneous!)'  and  as 
accurately  as  possible,  as  many  elements  of  ship  performance  as  possible.  Such  measure¬ 
ments  would  provide  an  overview  of  the  response  of  the  ship.  Many  practical  limitations, 
such  as  budget  constraints,  had  to  be  considered,  however.  In  addition  we  didn't  know  the 
real  limits  of  the  accuracy  of  these  kinds  of  measurements,  nor  did  we  know  how  to 
choose  the  proper  duration  of  observation  times.  After  lengthy  discussion,  it  was  decided 
to  record  each  type  of  measurement  for  three  minutes. 

Very  few  naval  architects  at  that  time  had  a  good  knowledge  of  probabihty  or  of 
statistics.  We  did  not  know  how  to  analyze  the  data  taken  at  sea,  and  we  did  not  know 
how  to  estimate  the  ensemble  characteristics  of  performance  from  a  single  sample  obser¬ 
vation.  The  simultaneous  records  of  the  averages  of  the  3-minute  responses  gave  us 
valuable  knowledge  of  actual  sea  performance,  but  we  realized  that  simple  averages 
sometimes  cover  or  conceal  important  information.  For  ship  oscillations,  of  course,  mean 
values  are  not  significant,  but  this  author  could  not  be  satisfied  with  merely  noting  the 
average  frequency  and  mean  amplitudes. 

During  the  test,  the  author^  was  responsible  for  measuring  the  ship’s  relative  speed 
and  developed  a  new  type  of  ship  speed  meter,  based  on  an  idea  of  Dr.  Shiha.  This  new 
meter  measured  the  frequency  of  Karman  vortices  produced  behind  a  triangular  cylinder 
towed  by  the  ship  and  gave  instant  indications  of  sp>eed  variations.  Although  the  towing 
point  was  selected  to  minimize  the  effects  of  pitching,  heaving,  and  rolling,  these  mo’ ons 
did  affect  the  speed  of  the  towed  body.  The  objective  was  to  eliminate  the  effects  of  all 
motions  except  surging  and  to  derive  from  these  records,  by  some  method,  the  real  varia¬ 
tions  in  the  ship’s  speed  relative  to  the  water.  The  author  later  found  that  an  advanced 
technique  of  multiple  input  analysis  would  have  been  useful. 

After  the  test  at  sea,  we  continued  to  look  for  methods  for  analyzing  the  data  and 
found  a  stochastic  process  analysis  used  in  weather  forecasting  to  predict  temperature, 
humidity,  precipitation,  and  so  on.  The  author  started  at  once  to  study  time  series  analysis 
and  tried  first  several  kinds  of  periodogram  analysis  and  then  correlogram  and  spectrum 
analysis.^  Because  of  the  poor  communications  in  the  engineering  and  scientific  fields  in 
Japan  at  that  time  (even  five  years  after  the  war)  we  had  very  little  information  about  the 
outside  world  until  1954  and  were  unaware  of  the  pioneering  work  of  Dr.  M.  St.  Denis 
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and  Prof.  W.  J.  Pierson  Jr.**  in  1953.  We  had  to  start  on  our  own  independent  of  the 
achievements  in  the  USA. 

1 .2  PURSUING  THE  IMPROVEMENT  OF  COHERENCY  FUNCTIONS 

After  the  bases  for  the  mathematical  and  statistical  theories  were  established  bs  N. 
Wiener,^'^  J.  L.  Doob7  C.  E.  Shannon,^  S.  O.  Rice,’  and  others,  mostly  in  the  1940's,  the 
stochastic  analysis  techniques  were  apphed  in  many  scientific  and  engineering  fields 
besides  communications  and  control  engineering.  These  techniques  were  adopted  rather 
early  in  the  analysis  of  ocean  waves  and  of  a  ship’s  response  at  sea.  This  method  has  been 
pretty  well  formulated  by  the  efforts  of  oceanographers  like  G.  Neuni»uic,'*^  W.  J.  Pierson, 
Jr.,*'  M.  S.  Longuet-Higgins,’"  D.  E.  Cartwright  and  M.  S.  Longuet-Higgins,'-  and  by 
the  pioneering  work  of  M.  St.  Denis  and  W.  J.  Pierson,  Jr.,'*  succeeded  by  E.  V.  Lewis, 
and  J.  F.  Dalzell  and  Y.  Yamanouchi,'^  and  is  now  rather  popular  with  us.  We  now  know 
that,  in  practical  applications  and  when  applying  certain  theories,  a  few  statistical  consid¬ 
erations  are  necessary  in  the  numerical  computations  in  order  to  get  reliable  results. 

In  Part  I,  the  author  tried  to  show  the  problems  encountered  in  sample  computa¬ 
tions,  in  the  so-caUed  correlation  method  that  are  also  closely  related  to  the  basis  for 
model  fitting  techniques  (that  is,  the  parametric  method  treated  in  Part  H)  and  reviews  the 
conventional  nonparametric  method.  The  author  did  not  intend  to  go  into  detail  about  the 
analysis  technique. 

The  coherency  function,  if  properly  calculated,  is  a  good  index  of  the  extent  to 
which  spectrum  analysis,  as  a  linear  process,  is  valid  for  application  to  a  stochastic  pro¬ 
cess.  A  few  results  of  this  author’s  efforts  in  this  field,  presented  later  in  Pan  I,  are  related 
to  improvement  of  the  techniques  for  obtaining  a  good  estimate  of  coherencies. 

1 .3  TIME  DOMAIN  CHARACTERISTICS 

Time  series  analysis  is  sometimes  called  spectrum  analysis,  thus  showing  that  esti¬ 
mation  of  reliable  spectral  functions  is  very  important  in  the  analysis  of  time  series. 

Spectral  functions  are  surely  powerful  functions  which  provide  good  information  on 
a  stochastic  process  and  also  resolve  the  tangled  relations  of  convoluted  types  in  the  time 
domain  However,  because  of  this  fact,  and  also  partly  because  (auto)  correlation  did  not 
appear  in  an  important  way,  in  St.  Denis  and  Pierson’s  pioneering  paper,**  rather  little  at¬ 
tention  has  been  paid  by  naval  architects  to  the  time  domain  relations  over  the  past  few 
decades.  From  his  first  involvement  in  this  study,  this  author  has  thought  that  time  do¬ 
main  functions  deserved  more  attention  and  has  made  some  efforts  along  this  line. 

Of  course  spectral  functions  and  correlation  functions  are  actually  the  same  function 
expressed  in  different  ways.  Sometimes,  however,  in  sample  computations,  applying  the 
time  domain  expression  helps  us  understand  the  characteristics  of  the  stochastic  process 
better,  because  we  are  accustomed  to  expressing  the  physical  process  by  differential  equa¬ 
tions  that  are  expressions  in  the  time  domain.  The  time  domain  expression  of  stochastic 
processes  helps  us  in  the  analysis  to  use  already  known  characteristics  of  the  process. 
Moreover,  the  correlation  window  operation  in  the  sample  computation  is  just  a  multiply¬ 
ing  operation,  whereas  the  spectral  window  operation  is  now  an  entangled  convolution 
operation  that  is  more  complex  in  real  sample  computations.  In  sample  computations. 
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consideration  of  the  windows  is  very  important  and  also  a  troublesome  problem  to  get 
reliable  results. 

Parametric  estimation  of  the  spectral  function,  explained  in  Part  n,  is  a  method  of 
fitting  a  certain  statistical  model  to  the  process  to  be  analyzed  and  then  estimating  the 
parameters  of  that  model.  This  method  is  based  on  time  domain  characteristics,  and  the 
model  fitted  is  closely  related  to  the  equations  of  motion  of  the  process  itself.  The  method 
represents  a  different  approach  to  spectrum  analysis  and.  this  author  beheves,  that  is  a 
promising  one.  supplementing  the  conventional  nonparametric  approach,  treated  m  Part  I. 
So.  in  Part  II.  several  types  of  discrete  parameter  models  will  be  introduced  in  detail,  and 
then  the  results  of  application  of  the  model  fitting  method  to  the  analysis  of  seakeeping 
data  will  be  shown.  By  this  method,  the  actual  response  system,  in  which  the  output  is  fed 
back  to  the  input  to  some  extent,  can  also  be  tackled.  This  kind  of  system  has  been  hard  to 
be  analyzed  by  the  conventional  method. 

1 .4  TREATMENT  OF  NONLINEARTTIES 

One  of  the  reasons  for  low  coherencies  in  linear  spectral  analysis  is  the  existence  of 
nonlinearities  in  the  response  or  in  the  input.  So  in  Part  HI.  the  treatment  of  nonlinearities 
in  process  analysis  is  reviewed.  One  of  the  greatest  achievements  in  this  field  for  the  anal¬ 
ysis  of  seakeeping  data  is  due  to  John  F.  Dalzell*^-’’  in  the  application  of  polyspectra. 

This  application  will  not  be  described  m  detail,  except  the  basic  idea,  of  this  treatment  but 
another  review  of  the  treatment  of  nonlinearities  will  be  given,  to  make  clear  the  mutual 
relationships  of  these  several  different  approaches  to  the  problem.  In  addition,  the  treat¬ 
ment  of  nonlinearity  in  response  characteristics  during  one  trial  has  been  introduced  in 
Section  3.4  in  Part  I,  as  an  example  of  multi-input  single-output  analysis. 

1.5  SCOPE  OF  STUDIES 

In  statistical  studies  on  seakeeping,  there  are  roughly  two  kinds  of  applications.  One 
is  based  on  the  invariant  characteristics  of  a  ship  itself,  and  its  behavior  is  studied  statisti¬ 
cally,  assuming  the  excitement  from  the  environment  is  also  stationary.  The  other 
involves  the  macroscopic  probabilistic  distributions  of  seakeeping  behavior,  assuming  a 
variety  of  changes  in  environmental  conditions.  The  former  is  sometimes  called  short¬ 
term  statistics,  and  the  latter,  long-term  statistics.  They  are,  of  course,  closely  related,  and 
the  short-term  statistics  are  usually  used  as  the  basis  for  studying  the  long-term  statistics. 
Here  mostly  short-term  statistics  will  be  treated. 
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PARTI 

A  REVIEW  OF  SPECTRAL  ANALYSIS  THROUGH  PERIODOGRAM 
(NONPARAMETRIC  SPECTRAL  ANALYSIS) 


To  clarify  the  problems  and  difficulties  encountered  in  sample  computations  and 
make  them  the  basis  for  further  discussions,  the  rough  scheme  of  the  techniques  of  spec¬ 
trum  analysis  (through  periodograms,  the  popular  nonparametric  method)  will  be 
reviewed  first  in  Chapter  2.  Then  some  ideas  proposed  by  this  author  for  solving  these 
difficulties  will  be  summarized  in  Chapter  3,  Parti.  Many  text  books, especially  the 
comprehensive  one  by  Priestley,^  were  used  as  references  in  Chapter  2. 

CHAPTER  2 


BASIC  PROCEDURE  OF  THE  SPECTRAL  ANALYSIS  AND 
THE  PROBLEM  OF  SAMPLE  COMPUTATIONS 

2.1  RANDOM  PROCESS  AND  ITS  CHARACTERISTICS 

Here  the  general  continuous  process  on  r  is  expressed  by  X(r),  its  realization  as  x(r). 
and  the  probability  density  distribution  function  related  to  this  process  as  pi(x).  The  gen¬ 
eral  discrete  process  is  Xt,  its  realization  is  x/,  and  probability  distribution  density  function 
is  pt(x).  Then,  as  expected  values. 


mean  [X(f)]  =  £  tX(f)] 


=  1 


xit)p,{x)cix  =  pit) 


(2.1) 


var. 


[X(it)]  =  £ 


[x(t)-pit)f  p,ix)dx 


aHt).  (2.2) 


Usually,  the  probability  density  distribution  function  p,(x)  is  a  function  of  t ;  accord¬ 
ingly,  the  mean  fi(t)  and  the  variance  a\t)  are  also  functions  of  time. 

Joint  probability  density  distribution  functions,  :JXi',X2,Xi,  .  .  .  x„)  exist 

for  all  n,n  =  1,2,.  .  .  n,(n  —  »)  and  all  p,,(xi),p,^(x2),  .  .  .  ;  p,,,j(xiX2) . ; 

Pi,i2t3(Xi,X2,X3'),  .  .  .  ;  ....  are  defined  as  their  marginal  functions. 
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TheoreticaUy,  if  all  the  joint  distribution  functions  of  all  orders 


.  .  .  . 

■^'2’ '^3)>  •  •  •  • 


Pi.t^h  ■  ■  ■  •  •  •  X„) . 

are  known,  the  probability  structure  of  X,  is  completely  specified. 

2.1 .1  Completely  Stationary 
If 


Phh  ■  ■ 


Xn)  —  Pij+k^t2+k,  ■  ■  • »  (-^l* -^2*  •  •  •  •  X„),  (2.3) 


for  any  rj,  r2 . t„  and  any  k,  this  process  is  completely  stationary. 

2.12  Stationary  Up  to  Order  m 

In  this  case,  the  joint  moment  up  to  order  m  should  be  the  same. 


£[(X(fi)r'(X(r2)}'"^ . (X(f„)r"] 

=  £[(X(ri  +  /:)r>{X(r2  +  k)p . (X(r„  +  (2.4) 

for  all  positive  integers  m^,  mi,  .  .  .  m, 
where 

mi+m2  +  .  .  .  .  m„  <  m. 

2.13  Stationary  Up  to  Order  2 

Especially  when  the  order  is  w  =  2,  the  process  is  called  weakly  stationary.  When 
the  probability  density  distribution  functions  are  Gaussian,  they  are  completely  deter¬ 
mined  by  the  means,  variances,  and  covariances  of  two  variables,  and  accordingly  they 
are  completely  stationary. 

2.1.4  Ergodicity 

When  the  ensemble  of  the  averages  across  the  processes  converge  to  the  corre¬ 
sponding  time  averages  along  the  process  over  period  N  (when  N  tends  toward  infinity 
and  the  mean  square  is  consistent),  the  process  is  called  ergodic.  The  ergodicity  is  a  more 
strict  condition  than  the  stationality  as  is  shown  in  Fig.  2.1. 
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EVOLUTIONARY 


STATIONARY 

STATIONARY 


Fig.  2.1 .  Stationarity  and  ergodicity. 


For  example. 


Rir)  =  F  [X(t)-Xit  +  T)] 

^00  r  00 

=  x(t)xit  +  r)p{x„x,+r)dxtdx,+r 

J  -00  J  -00 


1  c 


xit)xit  +  t)dt. 


(2.5) 


2.1  J  Summary  of  Gaussian  (Normal)  Probability  Distribution  Functions 

For  convenience,  the  Gaussian  probability  distribution  fimctions  for  various  mun- 
bers  of  variables  are  summarized  here.  For  a  single  variable. 


1  (x-fi,f 

2  al 


Normalized  by  2  =  (x-pfljoi. 


P(z)  = 


1 

JlJt 


e  2. 


(2.6) 


(2.7) 


For  two  variables  X  and  Y 
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p(x,y)  = 


2jrjcr;tr^l-g)-)) 


ITilTI  «P 


(x-fij-  2q(x  -  p,)(y  -  fij  (y-pj- 


(l-eVi  il-Q-)o^r  (l-e’ia; 


2jrA’/= 


-  exp 


aj,  2  IpojUY,  .  o: 


-(x-fi.Y 


(x-p,)(y-pr)  +  V  0 
A  A 


1 


2jrA'/ 


T  exp 


-  \  [^J.X-pYr  -  2fi.y{X-p,)(y-fir)  +  /SyyO'-M>-)') 


(2.8) 


Here, 


A  = 

Yxx 

Yxy 

cr^  QOxPy 

'S 

Yyx 

Yyy 

Q^xPy  Oy 

=  a^l-Q^a^j 

=  alaj{l-g\ 


(2.9) 


Vxx  =  •  X]  =  ai  ,  =  £:[X  •  y]  =  ga^y 

yyy  =  £[y  •  n  =  a2  ,  =  £[y  •  X]  =  ga^,.  (2.10) 


\Pxx  Pxy 

1 

1 

>* 

1 

1 _ 

1 

aj 

-QOj^y] 

- 1 

"  [yr‘ 

Yxy  Yxx 

-QO/Jy 

1 

(2.11) 


For  «  variables  Xi,X2, . X„, 


"  (2;rr/2Ai/2 


exp 


n  n 


i=i  f=i 


,  (2.12) 


PiXi,X2, 


where 


A  =  |x|  =  det(x) 

2:=  hi 


(2.13) 


and  a  ‘  is  an  element  of  inverse  matrix,  so  that 


. x„)  = 


U.14) 


2.2  PROPERTES  REQUIRED  FOR  ESTIMATOR 

The  following  properties  are  required  for  the  estimator  of  some  statistical  value. 

22.1  Unbiasedness 

As  the  number  of  samples  n  tends  toward  infinity,  for  the  estimator  6  of  real  6, 


bias(0)  =  [£(0)  -  ej  —  0. 


(2.15) 


22.2  High  Relative  Efficiency 

If  we  suppose  that  both  estimators  are  unbiased,  then  the  higher  the 

rel.  effic.  =  var.  (0i)/var.  {62) 

is,  the  better  estimator  d,  is  than  6^  • 

223  Small  Mean  Square  Error 

Mean  square  error  is  defined  as 

mHo)  =  £[(0  -  ef] 

=  £  {0  -  £(0)j  +  {£(0)  -  ej 

=  £  {0  -  £(0)j^  +  {£(0)  -  ef  +  2{£(0)  -  0 j£[e  -  £(0)] 

=  var.  {§)  +  b\e), 

_  A  -  A  A  A 

which  should  be  small.  WhenAf^(6i)  <  we  adopt  61  as  better  than  62  • 


(2.16) 


(2.17) 
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22.4  Consistency' 


As  the  number  of  the  samples  n  tends  toward  infinity  ,  var.  (0)  — ►  0,  and  bias 

0~(0)  0  must  be  satisfied.  Then  the  mean  square  error  M-(0)  -►  0  also  stands  and  is 

called  the  “mean  square  consistency.” 

22 J  Sufficiency 

_  A 

The  estimator  6  must  contain  all  the  information  Xi,  AS . A'„  in  thp  sample, 

relevant  to  the  estimator  of  6 


6  =  6  (Xi.Xi, - X„). 

2.3  AUTOCOVARIANCE  FUNCTION  AND  ITS  ESTIMATES 

When  the  process  X(r)  is  stationary  up  to  an  order  of  2,  the  covariance  function 

cov.  [X(0  X{r  +  T)]  =  £  [{X(t)-/r}{X(r  +  T)-//}]  =  £(t)  (2.18) 

is  a  function  ofr  only.  Then,e(T)  =  R{x)/R{0)  is  called  an  autocovariance  coefficient, 
and 

£(0)  =  £  [{X(r)-;rp]  =  var.  [X(r)]  =  (2.19) 

When  the  process  is  real  valued,  R{-x)  =  R{x)  as  in  Fig.  2.2,  and  when  the  process  is 
complex  valued,  R(-t)  =  R*(r).  This  function 7?(r)  is  a  measure  of  similarity  in  a  sense, 
and  is  also  a  measure  of  the  memory  of  the  process.  R{x)  plays  a  big  role  later,  in  the 
parametric  analysis  in  Part  II,  in  identifying  the  statistical  model  that  will  fit  the  process. 

Mean/i,  variance  a  R{x),  and  e(T)  are  constant  by  r. 


Fig.  2.2.  Autocovariance  function. 
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biased  by  (Irl/AO  R(r).  WhenN  -*  * ,  £[/?(r)]  -►  R{r),  and  is  therefore  asymptotically 
unbiased. 

By  statistical  mathematics,  we  can  get 

var.  [£V)]  ^  0(l/(/^-lrl))  (2.25) 


var.  [^(r)]  —  Oa/N). 


(2.26) 


A  A 

1.  When  r  is  small  relative  to  N,  the  difference  between  /?®(r)  and  /?(r)  is 

small,  and  the  bias  of  [£(/•)]  is  also  small. 

2.  When  r  becomes  large  relative  to  N  and  approaches  N-\ , 

bias  [£(r)]  -*  R(r).  However,  when  r  -►  » ,  R(r)  -*  0.  Therefore,  when 
N  is  very  large,  the  bias  remains  small  at  all  r. 

3.  When 


var.  [£°(r)] 

-  0(1) 

(2.27) 

var.  [Rir)] 

-  0(1/M). 

(2.28) 

A 

Therefore,  the  tail  of  correlation  R°{r)  shows  a  wild  and  erratic  behavior.  Also,  from 

A  A 

statistical  mathematics,  cov.  [f?(r)  R{r  +  s)]  was  computed  and  fairly  high  correlations 
between  neighboring  points  were  found  when  s  is  small.  When  r  tends  toward  infinity, 

A 

R{r)  -*  0.  It  can  be  concluded  here  that  R{r)  will  be  less  damped  than  R(r)  and  will  not 
decay  as  quickly  as£(r). 

2.4  SPECTRUM  ANALYSIS 

The  spectrum  function  decomposes  a  time  varying  quantity  into  a  sum  (or  integral) 
of  sine  and  cosine  functions. 

2.4.1  Spectrum  for  Various  Processes 
a.  For  Deterministic  Periodic  Functions — ^Fourier  Series 
For  periodic  functions  with  period  T, 
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1  * 
X(r)  =  -  +  X 

^  n=l 


(7int\  ,  .  [lint 

an  j  +  b„  sinl — 


(2.29) 


where 


1  r  ,  nnt  . 

=  —  X{r)  cos  —  dr 
t}  T 

_  T 
2 
T 

1  f  „  Tint  , 

=  —  X{t)  sm -  dr. 

tJ  T 

_  r 
2 


(2.30) 


The  a„  and  b„  functions  are  called  Euler-Fourier  coefficients.  Equation  2.30  is 
based  on  the  orthogonality  of  cos  ,  sin-^ . 

For  the  existence  of  a„,  b„,  and  for  the  convergence  of  the  series,  the  conditions 


T  T 


T  2 

I  \X{t)\dt  <  00  ,  I  \X{t)\^dt  <  00 

(2.31) 

J  J 

T  T 

"  2  *“7 

are  sufficient. 


Replacing  a„,  b„  by  co  =  Oo, 


gives 


the  total  energy  [-T/1  to  r/2]  as 


(2.32) 


and  the  total  power  [-T/2  to  r/2]  as 
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X 


(2.33) 


n=  — X 


ci  is  the  contribution  of  the  /ith  term  to  the  total  power,  as  in  Fig.  2.3. 


Fig.  2.3.  Power  spectrum  of  periodic  function. 

b.  For  Nonperiodic  Deterministic  Functions — Fourier  Integral 
For  nonperiodic  functions  with  finite  energy  as  in  Fig.  2.4, 


X{t)  = 


X 

I  X{0))  e‘^‘da). 
^00 


(2.34) 


14 


For  the  existence  and  convergence  of  this  expansion,  conditions 


I 

~00 


lX(r)l  dt  <  X, 


oc 

1 


{X(r)}‘  dr  <  oc 


—  QC 


are  necessary.  In  this  case,  Parseval’s  Relation  is 


(2-^6) 


j  X\t)dt  =  2jt  J  X(,w9d<j).  (2.37) 

—  oc  — oc 

Here,  as  in  Fig.  2.5  27rlX(a>)l‘iia>  is  the  contribution  to  the  total  energy  of  those 
components  inX(r)  whose  frequency  lies  between  o)  and  w  +  dto.  Accordiagly, 
2jt\X{o))\~  denotes  the  energy  density  function.  In  this  case  determining  the  power  makes 
no  sense,  because  when  T  -*  » ,  from  Eq.  2.36,  power  tends  to  0. 


Rg.  2.5.  Energy  spectrum  of  nonperiodic  function, 
c.  For  Stationary  Stochastic  Processes 

For  a  single  realization  x{t)  of  X(r),  as  in  Fig.  2.6  assume  £[X(r)]  =  0;  ATfr)  is  sto- 
chastj''ally  continuous.  [R(O)  =  a|,p(0)  =  1] 


X(t) 

1  jTV 

t 

A  A  A 

A  n  /\  A  JUr 

_ 

W  Vat 

1 

/V— ' 

Rg.  2.6.  Stationary  stochastic  process  with  finite  power. 
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Since  there  is  no  periodicity,  no  Fourier  series  exists,  and  because  of  stational- 

QC 

I  <  *  does  not  hold,  soX(/>  does  not  possess  a  Fourier  integral.  Thus. 


2n  lim  lX(a>)i*  will  not  exist  and  may  become  infinity.  So  here 


r 


Xiit)  = 


X(t) 

0 


-T<t<T 

otherwise 


(2.38) 


and  the  following  process  is  introduced: 


a 


Xjioi)e‘^'d(o 


—  00 


X-rioj) 


w  i 

=  —  I  Xj{t)e-^'dt  =  —  I  X(t)e-^'dt.  (2.39) 

2ji  J  2jt  j 


-T 


Then,  since  the  power  is  27rlX7<<o)l^/r  instead  of  2n:lX7<a>)l',  which  makes  T  -►  « . 
2jrlA:7<tu)l‘  . 

lim - — - is  for  the  total  process.  The  expected  value  of  this  power  is 


E  [Power]  =  lim  E 


2ji\Xii(or 

2T 


(2.10) 


and  is  defmed  as  s(.a)).  By  manipulation. 


2jdXj<a))\^  1  1  * 


OO  00 

=  ^  ^  I  j  XTit-r)e'^^'-^^dt 
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QD  '  OC 

=  -[  -f 

2jr  J  2T  J 


Xjir)  Xjir-r)dr 


cc  L  — oc 


e-^^-dr 


gc 

.±1 

2jr  J 


RTiT)e-’^^dt, 


♦Denotes  the  complex  conjugate. 


Here, 

I 


Rlix)  =  lim  —  j  Xjit'pCiit  -  T)dt. 
T-cc  2T  J 
—  00 


Accordingly,  for  the  total 

£  [Power]  =  sifo)  =  Iim£ 


2jt\X(co^\^ 

2T 


go 

=  lim^  f 

r— 00  2jt  J 


e-^'ElR7ir)]dr. 


Since  .Sf7<r)  was  defined  as  X(r)  for  -  T  <  t  <T,  and  otherwise  0, 


P7<r)  =  ^ 


1 

-1  ( 
2T  J 


Xit)  X(r-\T\)dt,  k\<T 


-iT-W) 

0 


'Ti>r 


I 

I 


(2.41) 


(2.42) 


(2.43) 


(2.44) 
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L 


E[X(r)  X(r■^■T)]  dt 


I 

--  I 

2T  J 


R{x)dt 


(r-iii) 


r 

Rit) 

1 

ii 

II 

27 

S) 

L  J 

As  T  approaches  infinity, 


iTi<r 

iTi  >  r 


(2.45) 


s{co)  =  lim 
r— 00 


i 

±f 

2jt  J 


irl 

1 - 

27- 


00 

■ii 


-T  t- 


e-‘^  R(t)dt. 


e-^^R{x)dx 


(2.46) 


For  the  existence  of  s{(o),  X(t)  must  have  an  auiocovariancefunction/?(T)  that  is 
continuous  everywhere,  including  at  r  =  0,  and 


00 

=  / 


/?(r)=  dS(Q}) 


(2.47) 


OO 

=1 


P(t)=  dF{(o). 


(2.48) 


This  relation  is  called  Wiener-Khinchine’s  Theorem.  Here,  F{w)  is  the  normalized 
integrated  spectrum  where  if  sifo)  is  continuous  and  smooth. 


m  w 

S(a))=  j  si(o)d(o  ,  and  F(ty)=  Jy(to)dco, 


(2.49) 


where /(m)  =  s{(o)/a% 
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The  properties  of  F{(o)  and  fio))  correspond  to  those  of  the  probability  density  and 
probability  density  distribution  functions,  P(tu)  and  p(a>),  for  continuous  distribution. 

2.4.2  Spectral  Representation  of  the  Stationary  Random  Process 

Instead  of  using  the  Fourier  integral,  the  stationary  process  is  more  generally  ex¬ 
pressed  by  the  Fourier  Stieltjes  integral  as 


go 

=  / 


X(/)=  mo)). 


(2.50) 


where  dZ(£u)  is  in  the  order  of  Oi/dta)  and  is  larger  than  dto.  dZifo)  and  dZ{u)  )  are  un- 
correlated,  that  is  they  are  orthogonal  to  each  other 

£  [ldZ(£t>)Pj  =  (2.51) 

When  the  process  has  a  purely  continuous  spectrum,  from  Eq.  2.49,  rewritten  as 


dS{(i))  =  s{(o)  dci). 


(2.49’) 


E  j^l<iZ(G))pj  =  si(o)da}  =  A5(a)). 


For  a  real  valued  process, 


R{-r)  =  R(t\ 


and 


si(0) 


w 

=-Lf 

2jt  J 


costUT  /?(r)  dx  =  —  cos  OUT  R(t)  dr. 
n 


1 


sifo)  is  an  even  function,  and 


s{-a))  -  s{<o). 


Therefore, 


00 

-\ 


/?(t)  =  j  cos  tor  j((o)  da  =  2  j  cosojt  s(co)  da. 

0 


-2]. 


(2.52) 


(2.53) 


(2.54) 


(2.55) 


(2.56) 
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2.4  J  Spectrum  of  Discrete  Parameter  Process,  Aliasing 
For  discrete  process 

1.  i?(r)  are  defined  only  for  integer  values  of  r,  r  =  0,  ±  I,  ±  2, 
Therefore 


5(£n)  =  ;l  f  e-^'' Rir, 

2jt  ^ 
r  =  -oc 


^  1  ® 
2jr  jr 


cos  rw. 


Here  Ar,  the  time  interval  of  the  discrete  process,  is  assumed  to  be  Ar  =  1 .  If 
Ar^  1,  changeoi  into  (o'  =a)/Af,  then 


(2.57) 


sico')  =  s(q))  Ar. 


(2.58) 


From  now  on,  in  this  lecture  note,  Ar  =  1  is  assumed. 

2.  s(q))  are  continuous  functions  but  are  defmed  only  for  frequency  -n  ^ 
(o  <  7t,  and  are  affected  by  aliasing. 


Rir)  =  dVifo)  r  =  0,  ±  1,  ±  2,  .... 


(2n+l)ff 


00  ^  00  ^ 

=  ^  I  e^'^dV{(o)=  ^  I  exp  [i((u  +  2n;r)r]  +  2wr) 

PI— _oo  ^  n=— 00  ^ 


(2/1-1  );r 


-  ^  I 

n=-oo  ^ 


exp  (icDr)  dV{w  +  2mt) 


i 

<  H—  _ Of 


dV{(0  +  2rm) 


dSi(o) 


(2.59) 


5(01 )  =  ^  V(ai  +  2wr), 


(2.60) 
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and 


^(0))  = 


dS(Q)) 

do) 


oc 

^  v((o  +  2jin). 
«=  -00 


(2.61) 


As  is  shown  in  Fig.  2.7,  at  frequency  two.  the  spectra  not  only  of  v((oo)  but  also  of 
v(too  ±  2jc),  v(a>o  ±  4jr),  v((0o  ±  6r) . are  folded  as  the  spectrum  5(ti>o).  as 


s((0o)  -  v(coo)  +  v(a>o  ±  2jr)  +  v(<uo  ±  4jt)  +  v(o}o  ±  6jr)  +  ...  . 

=  v(ti>o)  +  v(2jr  ±  fi>o)  +  v(4jr  ±  fl)o)  +  v(6jr  ±  a)o)+ .  (2.61’) 


This  is  also  illustrated  in  Fig.  2.8,  where  the  harmonic  functions  of  not  only  frequency 
1/5,  but  also  of  4/5, 6/5;  9/5, 11/5;  14/5,  16/5; ....  can  pass  through  the  same 
sampling  point  O,  as  marked  in  the  figtire,  at  Ar  =  1  sec  intervals  (see  Table  2.1). 
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Fig.  2.8.  Alieising  harmonic  curves  of  frequency  1/5,  4/5,  6/5; 
9/5, 11/5; .. .  can  pass  through  the  same 
sampling  points  at  Af  =  1 . 


Table  2.1.  Aliasing  frequencies. 


O) 

1 

0)0  =  Zrr  X  — 

°  5 

f  2n  -  0)0 

t  2n  +  0)0 

1 4n  -  0)0 
{  4n  +  0)0 

( 6ii  -  0)0 
(  6it  +  o)o 

.  .  . 

f 

fo  =  -°  =  ^ 

°  2n  5 

f  1  -  'o  = 

4 

5 

|2  -  fo  = 

£ 

5 

.  .  . 

1  +  fo  “ 

5 

1  2  +  fo  = 

11 

5 

Equation  2.61  or  2.61  ’  shows  that,  in  cases  of  discrete  process  analysis  of  the 
spectrum  ordinate  at  frequency  (Oo,  the  power  at  higher  frequencies  In  ±  o)o, 

An  ±  (Do . is  spiuiously  folded  on  the  real  power  attOo  as  in  Fig.  2.7. 

Accordingly,  when  a  continuous  process  is  to  be  transformed  into  a  discrete  process 
sampled  at  time  intervals  At,  aliasing  is  the  most  important  effect  to  be  considered.  The 
analyzed  range  of  the  frequency  spectrum  is  between  the  Nyquist  frequencies 

and  —  (whenAr=  1,  as  in  preceding  sections  -,7r  to  ;r  ).  Accordingly, 

n 

At  should  be  small  enough  to  avoid  aliasing.  Practically  it  is  advisable  to  take—  as 

At 

n 

large  as  —  >  (1.2  to  1.5)m«,  where  is  the  uppermost  frequency  of  the  interested 
At 

frequency  component  of  the  spectrum.  Then 
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Ar  <  - .  (2.62) 

(1.2  to  l.5)a)g 

2.5  COMPUTATION  OF  THE  SPECTRUM  FOR  A  DISCRETE  PROCESS 

Here  the  procedure  for  calculating  the  spectrum  for  a  discrete  process  is  summa¬ 
rized.  In  many  steps  of  the  computations,  the  finiteness  of  the  sample  process,  the 
discreteness  of  the  data,  and  the  choice  of  parameters  will  affect  the  results  statistically 
and  mathematically. 

25.1  Spectrum  Computation  Through  Periodogram 

First,  we  take  the  periodogram  PjyKto)  that  is  the  square  of  the  finite  Fourier  trans¬ 
form  Fjf(a>)  of  a  single  realization  of  this  discrete  process. 


= 


(2.63) 


Puio))  =  lFx(<u)l^  =  • 


2jtN 


1  1 


N 

e 

N  N 


rkot 


=  X  Z  -^6  ^2  COS  (fi  -  t2)a) , 


(2.64) 


/j— l  /2“i 


setting 


h-*t 
t2-*ti  +  r 

1  1 

=  7^  Z  T7  cos  rw . 


(2.65) 


From  Eq.  2.24,  we  know  that 


•  X,  X,+iri  is  the  biased  estimate  of  the  autocorrelation 


R(r).  Therefore, 


PNi(0)  =  — 
2Jt 


N-i 

^  Rir)  cos  ro). 
r  =  -(JV-l) 


(2.66) 
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For  the  entire  discrete  process. 


1 

E  [/'aKgj)]  =  —  J!  W'')]  cos  ro), 

2jt  ^ 

r  =  -(^-l) 


which,  from  Eq.  2.24 


2ji 


^-l 

S 


r  =  -(Af-l) 


R{r)  cos  ro). 


(2.67) 


(2.68) 


From  these  relationships,  the  spectrum  computation  from  the  periodogram  is  the 
same  as  the  spectrum  estimation  from  auto  correlation.  The  Fast  Fourier  Transform  gives 
the  periodogram  directly  through  direct  Fourier  transfer  and  is  statistically  the  same  as 
this  periodogram  analysis.  Accordingly,  care  should  be  taken  in  selecting  the  spectrum 
windows  to  be  used  in  the  FT'.T.  as  will  be  discussed  later. 

From  Eq.  2.68  if  N  becomes  » ,  surely 


E  -  sio))  .  (2.69) 

Then,  from  the  periodogram,  we  can  estimate  the  spectrum.  Pn{(o)  is  an  unbiased 
estimate  of  ^(<y).  Here,  however,  Rir)  is  the  theoretical  auto  correlation,  and  we  think  the 
spectrum  is  continuous  and 

n 

R{r)  =  J  s{(o)  cos  rwdo)  ,  r  =  0,  ±  1,  ±  2, .  .  .  (2.70) 

~7t 

Thus 


f  yt 

1  Irl  r 

[PaKo;)]  =  —  T  •  1 - -  I  s(u)  cos  rfi  dfi  cos  rw 

Til  j 

=  I  X  {A(-iH}~|cos  r{fi  +  (o)  + cos  r(/r-ty)|  dfx.  (2.71) 

^  r  =  -(N-l) 


Here  we  use  the  following  relations  on  summations  of  digital  quantities,  which  can 
easily  be  proved. 
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X  (N-lrl)^r)=  X  Z 

r  =  _(Af-l)  u=Or  =  -u 


in  (w  +  yj^J 


^  cos  =  — 

r  =  ~u 

i  j 

sin(|0) 

sin  («  +  y)^ 
u=c  sin(y^) 

>j  sin^^ 

sin^( 

(2.72) 


(2.73) 


(2.74) 


Then  from  these  relations. 


1  Irl  1  1 

—  y  <1 - -  cos  rd)  = - V  (N  -  Irl}  cos  np 

2?r  , w  ^  u 

r  =  -(Af-l)  r  =  -(A/-l) 


1  1  A'-l  "  1 

- y  T  cos  rtp  - T— ^ 


S  Fe,i(/>). 


(2.75) 


This  function  is  called  the  Fejer  function  FeiJ^O)  and  has  the  form  shown  in  Fig.  2.9. 


Ne 

When  — — ►  6, 


^  ,  rx  1  sin^0 

F,^(0),near  ^  =  0  ~  —  — y-v 


1  sin^0  N  I  sin^y 
kr  o/  fl  \  _  1  /I  I 


Fig.  2.9.  Fejer  Kernel  Function  Fgfj(e) 


Therefore, 


lim  = 

*0 


N 

—  lim 
2jt  9-*o 


IL 

2jt  ' 


(2.76) 


Putting  this  function  Ff^O)  into  Eq.  2.71  gives 


JT 

E  [Psioi)]  =  j  si<l>)^[FeJ,<t>  +  a>)  +  -0))]d(l>.  (2.77) 

-jt 


Taking  into  account  the  form  of  the  Fejer  kernel  Ff/  gives 


n 

=  I  s{<j>)Ft^{fj> -(o)  d4> . 

-3t 

When  iV  -*  00 , 


(2.78) 


Fe,  tends  to  Dirac’s  delta  function,  and 


(2.79) 
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(2.80) 


3t 

E  [/’M<w)]  =  [ s{<{>)  d{(p-(o)  dip  =  s{(i)). 

Af-»»  J 

-X 

When  N  is  finite,  £[/'jv{a>)]  is  a  kind  of  filtered  spectrum,  filtered  by  the  Fejer  kernel 
function 

Though  the  details  of  manipulation  by  statistical  mathematics  are  not  shown  here, 
we  know  that  the  variance  of  the  periodogram  can  be  calculated  as 

var.  )]  is  on  the  order  of  ^(lOp) ,  (2.81 ) 


(Op  = 


2jtp 

IT 


at  p  ^  0,— 


and  also  that  Pn((o)  follow  xl  <  distribution  with  degree  of  freedom  v  =  2.  As  the 

mean  and  the  variance  of  xl  are  v  and  2v  (here  2  and  4,  respectively),  then  the  signal  to 
noise  ratio  S/N  is 


S  mean  2 

—  ratio  = -  = - ^ 

N  standard  variation  (2  X 


(2.82) 


Equations  2.80  and  2.81  can  also  be  used  to  check  Eq.  2.82.  This  S/N  ratio  of  1  means 
that  this  value  of  Ps(fo)  is  a  very  poor  estimate  of  the  spectrum.  Further,  the  fact  that 

var.  [FaKw)]  does  not  tend  to  0,  even  when  N  tends  to  » ,  means  this  estimate  of 
F[Fjv((u)]  is  not  a  consistent  estimate  of 

Furthermore,  at  two  fixed  neighboring  frequencies a> land (W2,  cov  [Fjv(<Wi), 
Ps(o>2)]  can  be  calculated  to  decrease  by  the  increase  in  N. 

These  facts  are  reflected  in  the  erratic  and  widely  fluctuating  form  of  Psiio). 
Pn(o))  may  produce  a  spurious  peak  in  the  region  in  which  FaKoi)  is  large,  as  shown  in 
Fig.  2.10. 

2J2  Consistent  Estimation  of  the  Spectrum 
In  the  expression  of  the  periodogram. 
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Fig.  2.10.  Behavior  of  E[Pt^((£>)] . 


PfA.<o) 


J_ 

2jr 

r 


N-1 

I  «('•)  cos  ro) 
-(N-l) 


J_ 

2jt 


N-l 

•  RiO)  +  2  X  ^(r) 


cos  ro)  > , 


(2.83) 


var.  [i?(r)]  is  on  the  order  of  O 

andvar.  [/*A/(<y)]  is  on  the  order  of  0(1)  (2.84) 

are  known.  The  reason  for  the  large  var.  [P^iKc^)]  is.  as  shown  in  Eq.  2.83,  that  Fjv(co) 
adds  too  many  terms  in  ZP(r').  These  terms  are  slightly  correlated,  but  the  basic  effect  of 
too  many  terms  of  XR(r)  in  Eq.  2.83  remains  the  same. 

Accordingly,  the  way  to  reduce  this  large  variance  in  Fjv(af)  is  to  reduce  the  number 
of  additions  from  IV  to  A/  and  omit  the  term  N  »  M  in  Eq.  2.83, 

1  ^ 

E  [.So(ty)]  =  —  X  ^  cosro).  (2.85) 

2^  w 

This  reduction  decreases  the  variance,  and  from  Eq.  2.84  intuitively  we  find 
var.  [So(<u)]  is  on  the  order  of  0(M/7V).  This  can  be  proved  by  statistical  mathematics, 
although  the  manipulation  is  not  shown  here.  Substituting  Eq.  2.24  into  Eq.  2.85  gives 
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1  ^ 

E  [io(cy)]  =  —  2  ^  cos  rw 

^  r  =  -Af 


_1_ 

2n 


R{r)  cosro) 


sioj),  (2.86) 


as  M  tends  to  -*  » ,  but  more  slowly  than  N  tends  to  -►  «.  Namely,  N  -*  *,  Af  -*  « 

N  — 

but - -  00  also,  for  example  as  M  was  on  the  order  of  . 

M 

When  using  the  function  H’o(r),  Eq.  2.85  is  the  same  as. 


where 


io(tu) 


2jr 


(N-l) 

V  wo(^)  Eix)  cosro) 

r=-(N-l) 


(2.87) 


woCr)  = 


1 

0 


-N<-M  ^  r  ^  M<N 
otherwise 


(2.88) 


More  generally,  many  forms  of  functions  w(r)  besides  Eq.  2.88  are  proposed  as 

shown  in  Fig.  2.11.  /f(r)  cos  rto  are  even  functions  of  r;  therefore,  if  we  take  w{r)  as 
real  even  functions  of  r,  then 


1 

5((u)  =  —  y  w(r)  R{r)  (2.89) 

^  ,-W-l) 
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Function  w{r)  is  referred  to  as  the  lag  window. 


w{r).R(r) 

A 

R{r) 


Fig.  2  .11.  Lag  windows  w„(r)  with  R{r). 

253  Spectral  Windows 
From  Eq.  2.89, 


j  N-l 

S  *('■) 

^  r  =  -(W-l) 

As  Eq.  2.66  in  Section  2.5.1  shows. 


,  N-l 

PMof)  =  —  Z  Mr)e-^,  m  = 

2^  r  =  .(N-l) 


SO  inserting  these  relations  into  Eq.  2.89  gives 


I  dtp. 


si<o) 


I 


PNitp)  W{0)-<P)  dtp. 


(2.90) 


where 

,  N-\ 

2  Mr)  e-*.  (2.91) 

s(p))  is  obtained  as  a  weighted  integral  of  the  periodogram,  and  weighting  involves 
a  smoothing  operation  in  the  neighborhood  of  (D.  This  operation  reduces  the  contribution 
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from  the  “tail”  of  autocovariance /?(r),  which  shows  more  erratic  fluctuations  than  the 
real  value  of  R{r)  and  is  sutishcally  less  reliable,  as  discussed  at  the  end  of  Section  2.3. 
As  the  inverse  Fourier  transfer  of  Eq.  2.91 

W{<p)  dtp  {2Q2') 

r  =  0,  ±  1,  ±  2,  ...  ±  (N-  1);  [K’(r)  =  0  at  Iri  >N-  1]  . 

Sine?  wfr)  are  real  and  even, 


j  N-l 

^i<t>)  =  —  y  iv(r)  cos  r0.  (2.93) 

^  r.-tH-l) 

which  is  the  Fourier  pair  with  H’(r),  is  called  the  spectral  window  function,  and 
H'(r)  the  lag  window  function. 

given  by  Eq.  2.88,  is  one  of  the  lag  windows  and  is  called  a  “truncated 
window”  or  “do  nothing  window.”  Its  Fourier  transform  Wgip)  is  from  Eq.  2.73 


2jt 


M 


1 

y  cos  rtb  S6  — 


(2.94) 


This  function,  shown  in  Fig.  2.12,  also  referred  to  as  Diiichlet’s  kernel  function 
It  htis  a  high  peak  at  ^  =  0  and  rather  deep  valleys  on  both  sides  of  the  main 
lobe.  This  sometimes  causes  harm  to  the  computation  of  the  spectrum  and  results  in 
some  spurious  negative  values  for  the  spectrum  ordinates.  Mainy  smdies  have  been  made 
to  obtain  good  windows,  and  many  different  windows  with  different  characteristics  have 
been  proposed  and  claimed  to  be  good  from  different  points  of  view. 

Several  windows  dercribed  by  Priestley^  are  summarized  in  Fig.  2.13  and  Fig.  2.14 
in  the  form  of  pairs  of  lag  window  w(r)  and  spectral  window  W(<p). 

25.4  Ejfect  of  Windows 

Some  of  the  proposed  window  pairs  are  compared  in  Fig.  2.15.  They  were  designed 
mostly  under  the  following  conditions: 

1 .  Wf^p)  >  0  is  desirable  to  avoid  the  effect  of  large  negative  lobes. 
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Fig.  2.12.  The  Dirichlet  Kernel  Function  [wo(^)  =  Dm(^)]. 


2. 


R 

j  ^n{4>)  d4>-\,  i.e.,  H'(O)  =  1  from  Eq.  2.92,  to  keep/?(0)  unchanged. 

—iff 


3. 


3t 

j  d4> 


<  00 ,  to  make  s{q))  a  consistent  estimator. 


4.  0 1  uniformly  as  -•  » . 

N  fc 


5. 


0  as  N  -•  « ,  to  ensure  that  Wsi<f>)  is  not  too  narrow  in  relation 


to  ( —  j  and  to  have  s(co)  an  asymptotically  unbiased  estimator  of  j(tw). 


The  effects  of  the  spectral  windows,  most  of  them  designed  under  the  above 
conditions,  can  be  summarized  as  follows; 

1.  Bias,  that  was  0  for  the  estimator  “Periodogram  Pn{(o)"  is  now 


WINDOWS  -  1 


2M4-1 

2n 


Fig.  2.1 3.  Various  window  pairs  ( I ). 
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M 

n 


TRUNCATED  PERIODOGRAM 


77T 

"m 


W(4) 

I 


Fig.  2.15.  Various  spectral  windows. 

n 

bias  [i(ry)]  =  E  |i((u)  -  j(co)|  —  -j  j"(to)  j  0^W^(0)  do.  (2.95) 

-jt 


Here  s"ia))  is  the  second  derivative,  and  this  is  the  bias  at  the  peak  of  the  spectrum  when 
s'io))  =  0. 

2.  Variance,  that  was  ^{(o)  for  the  estimator  “Periodogram/V(o>)”  as  shown  in  Eq. 
2.81,  becomes 


A/  r  1  ^ 

var.  [5(m)] - 77 l^{u)da  =  —s^(.(o)  Y  H^(r),  (2.96) 

^  J  ^ 


where 

kiu)  =  =  vKr),  «  =  77 . 


(2.97 


When  w(r)  can  be  expressed  this  way,  the  window  is  called  a  parameter  window. 

3.  Equivalent  degree  of  freedom  v,  as  the  x~  distribution  that  was  v  =  2  for  the 
estimator  PnUo),  changes  into 


2N 

2N 

1 

00 

r 

k  4 

•  < 

M  1  l^{u)du  • 
—  00 

(2.98) 


4.  The  spectrum  window  blurs  the  real  spectrum.  Fig.  2.16.  Its  extent  depends  on 
the  window  bandwidth  (BW)  which  is  defined  in  several  different  ways,  although  every 
definition  hies  to  show  the  equivalent  frequency  range  over  which  the  smoothing  is  p)er- 
formed.  For  a  single-beaked  spectrum,  this  blur  depends  on  the  relative  bandwidths  of 
the  specmim  and  the  spectrum  window.  Specmim  bandwidth  is  defined  as  the  frequency 
range  in  which  the  spectrum  shows  half  of  the  peak  of  the  spectrum. 


Rg.  2.16.  Blur  o1  the  spectrum  by  Daniel’s  window,  wthere  the 
bandwidth  is  2idM. 

The  resolution  will  be  lower  as  the  window  bandwidth  becomes  narrower. 
2  J_5  Expression  of  Confidence  Interval  and  Precision  ofsico) 

1-  ®y  Jfv  Approximation 
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As  was  mentiooed  in  the  preceding  subsections,  i(a>)/s(a>)  can  be  approximated  by 
distribution,  with  degree  of  freedom  v.  Equivalent  degree  of  freedom  v  is  expressed 
by  Eq.  2.98  when  the  window  is  of  the  parameter  type. 

•  percentage 

points  of  xl  distribution,  as  shown  in  the  key  in  Fig.  2.17, 


Accordingly,  if  a^ia)  and  bv(a)  show  the  lower  and  upper 


ioo(f 


Prob.  \x;  ^  (hia)]  =  Prob.  \xl  >  by{_a)\  = 


Prob.  [Ovia)  <  <  bv(o:)]  =  <2- 


1  -a 


(2.99) 


(2.100) 


Therefore,  a  %  confidence  interval  is 


V 


s((o)  , 


V 

Ovia) 


sio)) 


(2.101) 


This  range  is  shown  in  Fig.  2.17  as  a  function  of  v,  witha,  the  confidence  level,  of 
80%,  90%,  or  95%  as  a  parameter. 

2.  Approximation  by  Normal  Distribution;  see  Fig.  2.18. 

We  know  that  when  N  tends  to  infinity,  R(r)  follows  the  normal  distribution,  and  as 
s(o))  is  a  linear  combination  of  R{r),s{(o)  also  tends  to  follow  the  normal  distribution. 
This  relation  can  be  assumed  also  from  the  fact  that,  when  degrees  of  freedom  v  — »  «>, 

as  fy  — ►  »  ,  ^^-distribution  tends  to  the  normal  distribution.  Then,  using  the  normal 
distribution,  we  can  get  the  other  expression  for  the  confidence  interval,  as 


si(o)  ±  c{x)J\ai.  s{q)). 

Equations  2.100  and  2.101  give  the  confidence  interval  as, 


1 + c(a)  yj 


s((o)  , 


l-c(a)VT 


i((o) 


(2.102) 


(2.102’) 
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v/bv(a) 


100 


Fig.  2.17.  Confidence  band  by  x^istribution. 


This  interval  is  shown  in  Fig.  2.19  at  a  confidence  level  of  90%  (c  =  1 .64)  as  an 
example,  with  values  of  thex'-distribution  at  the  90%  level.  As  v  increases,  the  confi¬ 
dence  band  shows  almost  the  same  level  as  the  one  from  thex^-distribution. 

3.  Expressions  of  Precision  of  the  Spectrum  Estimation 

There  are  several  ways  to  express  the  measure  of  precision  of  the  spectral  estimates, 
and  they  are  related  to  each  other  as  shown  in  Priestly.”^ 
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S(CC))-C  (a)V  VAR.S(CD)  S(CO) 


s(co)  +  C  (a)  VVAR.  s(co) 


Fig.  2.18.  Approximation  by  normal  distribution. 

i.)  p%  Gaussian  range  of  percentage  eircH-  6(a)) 

t.  X  .V  v(ty)  \b(a))\ 

6(0))  =  dp)  — -  +  — —  , 

s((o)  sio>) 

where  v(<w)  « [vjr.  standard  deviation 

h(<u)  =  £{i(<o))-s(to):  bias 

c(p)  is  the  two-sidedp%  point  of  the  standardized  normal  distribution, 


l5(m)  -  .y(Q;)l 
5(<y) 


S  d(Q)) 


P 

100 


ii.)  The  mean  square  percentage  error  i/iw) 


T((o)  = 


[£{i(a>)  -  5(a>)p]  ^2^^^  ^ 


S^((0) 


By  Chebycheff’s  inequality. 


Rg.  2.19.  Confidence  band  by  x^istribution,  and 
Gaussian  distribution  (90%  level). 


in  particular  if  we  ignore  the  bias 


b{(o)  =  0,  (5(fl>)  =  cip)rj{Q)) . 

iii.)  The  signal  to  noise  ratio  introduced  by  Parzen^  is 


(2.106) 


(2.107) 
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[var.  (s(aj)}j 


1/2 


If  we  use  the  asymptotic  normality  of  s(ai),  for  large  N 


(2.108) 


i(m)-£(i(<u)) 


So  if  we  set 


£(i(m)) 


3(0))  =  c(p) 


<  6 


P 

100 


var.  (5(01  )}j 


1/2 


5(0)) 


(2.109) 


(2.110) 


and  omit  the  bias  b((o)  in  Eq.  2.103,  i.e.,  if  b(a>)  =  0  or  £[r(a))]  -*  s(a)),  then 


6(0))  =  c(p)- 


(2.111) 


SNRjiCo))}  ■ 

This  relation  shows  that  SNR  |s(a>)|  is  a  simplified  version  of  d(a>),  when  the  bias 
of  s(ai)  is  ignored. 

Also  SNR  can  be  expressed  by  equivalent  degree  ot  freedom  v,  defined  by  Eq.  2.98 


as  V  = 


2N 


^>v^(r) 


,  and  as  s(ai)  follows  the  ^^-distribution  with  degree  of  freedom 


V,  where  the  mean  and  variance  are  v  and  2v, 


or 

V  =  2[SNR{j(<w)}]2  (2.112) 

FromEqs.  2.111  and  2.112, 


(5(<o)  =  c(p)j2/v. 


(2.113) 
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The  relationship  shows  that,  if  we  assume  the  asymptotic  normality  of  the  spectrum 
estimate,  then  the  proportional  error  <5(0)  can  be  estimated  from  the  equivalent  number  of 
degrees  of  freedom  of  the  spectrum  estimation  v,  by  Eq.  2.1 13,/?  and  c(p}  being  the  level 
of  confidence  and  p/100  the  point  of  normal  distribution  as 


c{a) 

I 


1 - 

-=^e  2  dx  = 

v27r 


~c{a) 


P 

100  ■ 


(2.114) 


From  Eq.  2.98  or  Table  2.2,  we  know  that  v  is  proportional  to  N/M,  and  for  most  of 
the  windows  2  to  3  times  N/M.  If,  for  example,  N/M  »  10  to  15,  then  v  is  on  the  order 
ofv  =  30.  If  we  adopt  the  confidence  level  of  p  =  0.95  (95%),  then  c(p)  =  1.96.  From 
the  appropriate  normal  distribution 


<5(m)=  1.9672/30  ^  0.506. 

This  value  means  that  the  estimate  of  the  spectrum  ordinate  has  a  proportional  error  in  the 
order  of  50%  for  this  example. 

2.5.6  Choice  of  the  Spectral  Window 

It  is  now  clear  that,  in  order  to  get  a  consistent  estimate  of  s(a>),  we  have  to  use  a 
spectral  window.  Several  spectral  windows  and  their  effects  on  the  estimation  are  summa¬ 
rized  in  Table  2.2,  from  Priestley.^ 
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Table  2.2.  Effect  of  spectral  windows. 


CHARACT. 

EXPONENT 

BIAS 

bM 

VARlSto))) 

s2(a)) 

EQUIV 
DEG.  OF 
FREEDOM 

V 

WINDOW 

BANDWIDTH 

Bw 

TRUNCATED 

PERIODOGRAM 

00 

0(M-®) 

0:  DECAY 
INDEX 

2“ 

N 

N 

M 

BARTLETT 

1 

-  i  s-(b» 

1.5  “ 

N 

3  N 

M 

1.56  — 

M 

DANIELL 

2 

1.65  —  s'im) 

m2 

M 

N 

2li 

M 

m 

TUKEY 
—  GENERAL 

2 

a  —  s‘{(£>) 

m2 

2(l-4a  +  6a2) 
M 

X  — 

N 

(1-4a  +  6a2)-’ 

xii 

M 

M 

TUKEY 
—  HAMMING 

2 

0.23  —  s-«o) 
M2 

0.80  — 

N 

2.52  — 

M 

2.45  — 

M 

TUKEY 
—  HANNING 

2 

0.25  —  s-((o) 
M2 

1.33  M 

N 

2.67  ii 

M 

2.35  fL 

M 

PAR2EN 

2 

0  01  ^  s*(C0) 

0.54  M 

N 

3.71 

M 

3.82  — 

M 

BARTLETT 
—  PRIESTLY 

2 

0.99  —  s-«o) 

m2 

1.20  M 

N 

1.4  N 

M 

1.55  - 

M 

The  spectral  windows  that  make  the  spectral  estimate  consistent  also  affect  the 
estimate  in  several  other  ways,  as  was  summarized  in  Section  2.5.4.  The  most  important 
parameter  of  this  window  is  the  size  of  M,  which  determines  the  window  bandwidth 
(BW),  that  is  by  Priestley^  expressed  as. 


B 


W  ” 


where 


=  lim 

u-*0 


i-m 

\u\q 


(2.115) 


(2.116) 
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q  is  the  largest  integer  that  is  a  characteristic  exponent  of  the  functioD 


m 


■<ih 


w//(r),  and  the  value  of  q  is  listed  in  Table  2.2  for  each  representative 


spectrum  window.  When  ^  =  2,  as  is  the  case  for  most  of  the  listed  spectral  windows, 
there  is  a  relationship 


bifo)  is  on  the  order  of  ^s'\(o)  |  <p^W{<f))d<f> 


!■ 


-71 


is  on  the  order  of - ^s”{q}) 

2M~ 


OD 


K{(p)d(i), 


(2.117) 


where 


I 


.00 


k{u)e~^“du 


or  ki.u)=  J 
.00 

To  state  the  relative  bandwidth  of  the  spectral  windows,  we  must  define  the  bandwidth 
of  the  spectrum  itself.  UsuaUy,  B,  =  <U2  -ft>i  where  a)2ai>da>i  are  the  frequency  points  at 
which  the  spectrum  shows  half  the  maximum  power  of  the  spectrum  peak  as  in  Fig.  2.20. 


Rg.  2.20.  Spectrum  bandwidth. 
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To  keep  the  blurring  effect  of  the  spectral  window  small,  the  bandwidth  By,  of  the 
window  must  be  small  compared  with  the  spectrum  bandwidth  Bs-  By  =  is  a  popular 
guide. 

As  explained  in  Sections  2.5.2-2.5.5  the  following  tendencies,  as  shown  in  Table 
2.3,  are  now  clear. 

Table  2.3.  Statistical  effects  of  the  change  of  M. 


M 

VAR. 

RELIABILITY 

BIAS 

WINDOW-BAND  WIDTH 

RESOLUTION 

LARGE 

URGE 

DOWN 

SMALL 

SMALL 

UP 

t 

t 

t 

t 

t 

t 

SMALL 

SMALL 

UP 

URGE 

URGE 

DOWN 

From  sampling  theorem  it  is  known  that,  if  is  calculated  from  the  frequency 

interval  Atu  =  — ,  i(a))  is  completely  determined,  although  since  r((u)  is  a  continuous 
M 

function,  it  would  be  quite  in  order  to  evaluate  it  over  a  much  finer  set  of  points.  It  has 

been  shown  that  we  can  generally  use  the  value  Aa>  =  for  Daniel’s  window,  where 

In  n  71 

By,  =  — ;  accordingly  Am  =  — .  When  we  use  Am  =  — ,  the  spectrum  window  effect 
MM  M 

will  be  replaced  by  a  weighted  mean  of  the  ordinates, 

M 

CCiPM 

Values  of  a,  for  several  windows  are  shown  in  Table  2.4. 

This  table  also  gives  values  for  the  coefficient  proposed  by  H.  Akaike,^^  for 
minimizing  the  bias  and  variance  of  the  estimate  for  the  spectrum  window.  W2  or  its 
simplified  form  Q  is  generally  used  in  most  of  our  work,  although  W3  would  give  a 
better  spectrum  with  steeper  peaks  and  deeper  valleys.  As  a  general  procedure  for 
choosing  the  best  of  Wj  to  W3,  Akaike  advises  trying  them  all  and,  if  no  improvement 
in  spectral  form  is  recognized,  adopting  the  window  of  the  lower  order. 


i(m)= 


( 

m  +  t —  . 

\  m) 


(2.118) 
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Table  2.4.  Coefficients  a,  for  various  windows. 


°-3 

«-2 

“-1 

“0 

“1 

“2 

^3 

TUKEY 

GENERAL 

a 

1-2a 

a 

HAMMING 

0.23 

0.54 

0.23 

HANNING 

0.25 

0.50 

0.25 

PARZEN 

0.008 

0.164 

0.657 

0.164 

0.008 

AKAIKE’S 

-W, 

0. 

0.2434 

0.5132 

0.2432 

-W2 

-0.0600 

0.2401 

06398 

0.2401 

-0.0600 

-  W3 

0.0149 

-0.0891 

0.2228 

0.7029 

0.2228 

-0.0891 

0.0149 

—  Q 

-0.06 

0.24 

0.64 

0.24 

-0.06 

WIDE 

NARROW 


2-5.7  Use  of  the  Fast  Fourier  Transformation  (FF'.T.)  Method 

There  are  misunderstandings  sometimes  that  spectrum  analysis  by  the  Fast  Fourier 
Transformation  (F.F.T.)  technique  is  completely  different  from  getting  the  spectrum  from 
analysis  of  auto  correlation  functions.  However,  Eq.  2.64  shows  that  this  method  is  mere¬ 
ly  the  one  that  gives  the  spectrum  through  a  periodogram,  and  Eq.  2.66  shows  that  it  is 
the  same  as  calculating  the  spectrum  from  the  Fourier  transform  of  the  auto  correlation 
function. 


Tjvp 

Through  the  F.F.T.  we  get  the  spectrum  ordinates  at  frequency - ,  or  in  the  range 

N 

2jw  N 

-7t  ^  CO  <  n,  at  the  frequency  points  p  =  0,  ±  1,  ±  2,  ...  ±  ~2'^  thedis- 

N 

Crete  case.  This  procedure  corresponds  to  getting  +  1  ordinates  in  the  frequency  range 


0  to;r,  or  AT-*- 1  ordinates  in  the  frequency  range  -n  to  n.  It  is  the  same  as  using 
N 

M  =  —  for  a  rectangular  spectrum  window  and  as  a  result,  the  individual  spectrum  ordi- 
N 

nates  of  ^-1- 1  are  so  unreliable,  statistically,  and  th^  variance  so  large  that  SNR  =  100%, 

equivalent  to  a  degree  of  freedom  v  =  2.  Accordingly,  the  same  considerations  on  the 
use  of  the  spectral  window  as  were  necessary  for  auto  correlation  methods  are  necessary 
for  the  FT.T.  method  after  getting  the  “raw”  ordinates  at  N  +  1  frequency  points.  Com¬ 
mercial  programs  or  even  a  speciahzed  spectrum  analyzer  through  F.F.T.  are  now 


available,  but  sometimes  they  do  not  say  anything  about  “windows.”  We  have  to  be  care¬ 
ful  in  our  choice  of  the  wmdow  to  be  used  in  the  analysis  to  obtain  a  reliable  spectrum. 

We  can  cut  the  number  of  computations  from  the  order  of  A'*  for  autocorrelation 
methods,  N  being  the  number  of  data,  to  N(r\  +  r2  +  •  •  •  for  the  F.F.T.  method  when 

N  is  factored  toN  =  rj  ■  ri  •  .  .  .  .  r^.  Very  commonly,  N  =  2^  is  used  and  then  the  num¬ 
ber  of  computations  is  on  the  order  of  2pN  for  F.F.T.  For  example,  when  N  =  1024  =  2“^, 
the  number  of  computations  is  reduced  from  the  order  of  1 ,000,000  to  20,000,  or  about 
1/50. 

The  spectrum  function  i :  a  powerful  expression  for  showing  the  characteristics  of  a 
time  series.  However,  we  sometimes  need  the  autocorrelation  function  to  find  the  proper 
pair  of  lag  and  spectral  windows,  to  decide  the  size  of  M  compared  with  V,  or  to  find  an 
adequate  statistical  model  from  which  we  can  go  to  the  parametric  analysis  of  the  pro¬ 
cess,  as  will  be  mentioned  in  detail  in  Part  n.  Chapter  5. 

The  autocorrelation  function  can  be  obtained  as  the  Fourier  transform  of 
s{(o)  or  as  the  periodogram  PNiw)  where,  from  Eq.  2.90, 


oo 

=/ 


s(a})=  PfA<p)  W^ico -^)  d^. 


I- 00 


From  Eqs.  2.64,  2.63,  and  2.66, 


(2.64’) 


(2.63’) 


1 


AT-l 


PNi(Op)  =  —  X  kr)e-^^. 

^  r  =  -(A/-l) 


(2.66’) 


N 

Analogously,  from  Eq.  2.64’  or  from  Eq.  2.66’,  we  see  that  from  y  ordinates  of  it 

A  A  N 

is  impossible  to  get  N  values  of  /?(r),  r  =  1,  .  .  .  N.  We  get  only  f?(r)  values  at  r  <  -j . 

A 

In  order  to  get  R(r)  values  at  r  =  0  to  AT,  we  need  N  ordinates  of  the  spectrum  or  the 
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periodogram  Pf/itUp)-  To  make  this  possible  we  need  N  more  data,  and  that  can  be  real¬ 
ized  by  adding  N O’s  to  the  original  data,  as  aco . 0. 0, 0 . 0 . 


2N 


In  this  way  the  FJ^.T.  method  can  also  be  used  for  computing  the  correlation  function 
from  the  spectrum  ordinates  at  N  frequency  points  as  in  Fig.  2.21 . 


Fig.  2.21.  Frequencies  to  calculate  the  spectrum. 


2J.8  Filtering 

As  was  mentioned  in  Section  2.4.3,  in  sampling  a  continuous  time  series  for  compu¬ 
tation  of  the  spectrum,  we  have  to  pay  attention  to  aliasing,  and  the  sampling  interval  Ar 
should  be  small  enough  to  avoid  the  aliasing.  We  must  also  be  careful  about  the  leakage 
of  power  or  the  blurring  effect  through  the  spectral  window,  especially  when  the  spectrum 
has  sharp  peaks  or  steep  valleys,  because  the  spectral  window  acts  as  a  smoothing  filter. 
Besides,  sometimes  we  are  especially  interested  in  the  spectrum  over  a  certain  range  of 
frequencies.  Then  the  filtering  technique  is  helpful  in  dividing  the  power  of  the  spectrum 
by  the  frequencies  or  in  modifying  the  shape  of  the  spectrum  to  a  shape  more  easily  han¬ 
dled. 

Generally,  for  a  continuous  process,  the  filtering  effect  can  be  expressed  as 

00 

r(f)=  I  g{r)X{t-r)dt,  (2.119) 

•  00 

where  X(r),  Y(t)  are  the  original  and  filtered  processes,  respectively,  and  g(r)  shows  the 
filtering  effect  in  the  time  domain.  For  a  physically  realizable  filter  g(T)  =  0,  for  r  <  0,  the 
range  of  integral  can  be  from  0  to  <» .  Then  from  the  discussion  in  the  preceding 
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subsections  and  assuming  the  existence  of  a  specmun  of  X(r).  y(r)  and  a  Fourier  trans 
form  of  g(u). 


SYYidf)  =  ^G(a))\~sxxi(J^) ,  (--1 

00 

G(m)=  j  g(r)e~'*^dt.  (2.i: 

>00 

A  few  samples  of  simple  and  ideal  filters  are  given  as  follows: 

1 .  Band-pass  filter  (Fig.  2.22) 


G  (oj) 


Rg.  2.22.  Band-pass  filter. 


Then 


IG(<o)P  = 


1 

0 


for<yi  <  kol  <  0)2 
otherwise 


g{r)  =  ^  j  Gi(o)e‘*“‘do) 
>00 


e^'do) 


(2.i: 


sin  <021'“  sin  (WiT 

7t  X 


for  all  T 


(2.i: 


From  the  form  of  this  function  and  Eq.  2.119,  we  need  the  input  process  X{t)  for 
-00  <t  <  00 ,  so  theoretically  this  filter  is  physically  unrealizable. 


2.  Low-pass  filter  (Fig.  2.23) 


Rg.2  .23.  Low-pass  fitter. 
When  0)1  =  0  for  a  band-pass  filter. 


f 


IG(q))|2  = 


1 

0 


l0)l  <  0)0 
lfl)l  >  0)0 


(2.124) 


sino)oT 

nx 


3.  High-pass  filter  (Fig.  2.24) 


(2.125) 


1  4  G  (®) 

1.0 

Wn  1 

0  u 

0  _ _ 

Fig.  2.24.  High-pass  filter. 
When  0)2  -*  »  for  a  band-pass  filter. 


IG((u)12  = 


0 

1 


V 


IO)l  ^  0>o 
l0)l  >  0)0 


Also  for  digital  processes,  we  can  express  the  digital  filtering  by 


(2.126) 
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(2.127) 


k 


Y,=  Y.SrX,-r. 

T  =  0 


SYYi(i>)  =  \G{a))\^sxx(o}) 


k 

Giw)  =  X  Sr 


r  =  0 


^-UOT 


4.  Low-pass  digital  filter  (Fig.  2.25) 


Fig.  2.25.  Low-pass  filter. 

For  example,  the  simplest  equation  for  this  type  will  be 


rr  = 


1 

2 


(X,_i  +  X,). 


Then 


SYiico)  -  IG((0)fsxx((0) 


-ll+e-^i 

2 


Sxxito) 


=  ■^{1  +cos<o}5xx<a>). 


5.  High-pass  digital  filter  (Fig.  2.26) 


(2.128) 


(2.129) 


(2.130) 


(2.131) 
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Fig.  2.26.  High-pass  filter. 
The  simplest  equation  for  this  type  will  be 


Then 


SYY  = 


2 


sxx(^) 


=  -j{l-cosa>)5xx(a)). 


6.  Pre-whitening  digital  filter  (Fig.  2.27) 


Fig.  2.27.  Pre-whitening  filter  |syv(a))/Sx)f(o))|’^. 
If  a  digital  filter  such  as 


Yi  =  ooXt  +  aiXt^i  +  a2Xt-2  ■  ■  ■  ■  ■  ■  , 


(2.132) 


(2.133) 


(2.134) 
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is  used  and  the  spectrum  syj<(y )  becomes  a  white  spectrum  sj7<a>)  =  Of,  then  as  will  be 
shown  in  Section  5.4.2  of  Part  II, 


snico)  =  Oy  =  lao  +  aie +  .  .  .  +aie~^l^sxxio^)-  (2.135) 


Then 


sxx(^)  = 


_ g? 

lao  +  t3ie~^  +  a2e~^'^+  . 


.  .  +ajte 


(2.136) 


If  we  can  find  the  filter  that  will  make  this  spectrum  completely  white,  that  filter  is 
called  a  complete  pre-whitening  filter.  Then  fi-om  Eq.  2.136,  we  can  find  sxx(tw)  from 

the  variance  Oy  of  the  filtered  process  (that  is  uniform),  and  the  frequency  function 

IG(m)P  =  lao  +  +  a2e-^  +  .  .  .  +  •  (2-137) 


This  tells  us  that  finding  the  pre-whitening  filter  G(q))  is  the  same  procedure  as  fit¬ 
ting  a  model  to  the  X,  process  expressed  as 

6/  =  aoX/  +  aiXj_i+  ....  akXt^+  .  ,  (2.138) 


where  €,  is  a  completely  random  process  with  variance  ay.  This  is  the  problem  of  AR- 
model  fitting  to  the  process  X(f),  whinh  will  be  discussed  in  detail  in  Chapter  5,  Part  II  of 
this  paper. 

2.6  MUm-VARIATE  SPECTRAL  ANALYSIS;  SPECTRAL 
ANALYSIS  OF  FREQUENCY  RESPONSE 

2.6.J  Two  Variate  Spectral  Analysis 

If  there  are  two  stochastic  processes  {Xi^}  {X2^}  r  =  0,  ±  1,  ±  2 . each 

weakly  stationary,  thenCov.  X2^j}  is  a  function  of  (ri  -r2)  only. 

For  this  stationary  bivariate  process,  the  correlation  matrix  is  defined  as 


where 


R{r)  = 


Ru(r) 

Riiir) 


Rnir) 

Rllir) 


(2.139) 
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Rn(f)  =  E 

R22(X)  =  E  \X2j-IX-^  {X2./+r-//2) 
R2\ir)  =  E\Xij-fii]"[X2j^r-ti:^  .. 
R\2ir)  =  E  (-^u+r-/Ui)  .  . 


Xj  leading  AS  (2.140) 


X2  leading  Ai 


^ii(-^)  =  ^ii(^) 


Rnir)  =  Rlii-r) 
♦Denotes  the  complex  conjugate. 


ei  !('■)  = 


Q22(r)  = 


Q2\ir)  = 


Rn(0) 

Rllir) 

^22(0) 


R2\{r) 


(/?ll(0)-/?22(0)p/2 


Then  the  spectrum  matrix  is 


(2.141) 


(2.142) 


s(ft})  = 


5ll(o)  Si2i(0) 
S2i(Q))  S22(C0)  ’ 


where 


(2.143) 
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1  ” 

^Ii(w)  =  —  y  Ruir)e-‘ 

2jt  ^ 

r  =  -oc 


-irw 


'  r  =  -oc 

1  * 

^22(CO)  =  —  X 

iJl 

r  =  -ac 

1  * 

S2l((i>)  =  ~  X 

r  =  — 00 


.  .  leading  Xt 

r  =  — 00 

^I2(<u)  =  :;^  X  Rnir)^'*'^ . ^2  leading  Xi. 

In  ^ 


(2.144) 


For  the  spectnun  to  exist,  the  correlation  function  must  be  absolutely  summable,  for 
example, 


X  •^2i('')l  <  *  • 


(2.145) 


By  spectral  representation,  using  the  Fourier-Stieltjes  form. 


7t  n 

V-  _  I  (2.146) 


Xi.,  =  j  e"^dZii(0)  X2,t  =  j  e‘'^dZ2i(0) 

— iTT 


E[dZ[iQ})  dZiioj’)]  = 


0,  0)  ^  (o' 

S\\((o)d(o,  (0=0)' 


(2.147) 


£[dZ5(<u)  dZ2(<w')]  = 


0,  Q)  ^  (o' 
S22{<0)d(0,  (0=0)’ 


(2.148) 


dZi{(o),dZi{a)'y,  dZ2i(o),dZ2{(o')  are  orthogonal,  respectively,  and  also  cross  orthogonal. 


E[dZ*i{(o)  dZ2{(o')]  = 


0,  (o  ^  (o' 

S2\((0)d(0,  (0=0)' 


(2.149) 
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(2.150) 


£[^/z5(£o)  dZiio)’)]  =  I 
Inverting  Eq.  2.144  gives  for  the  two  variate  process 


0,  0)^0)' 

Si2i.(o)dw,  a)=(o' 


Ri]{r)  =  I  e'’^S2\{(ji})d(o,  Rn(.r)  =  j  e""5i2(co)^/w), 

-n 

as  was  for  the  single  variate  process 


n  n 


£ll(r)  =  I  e" 


7t  7t 

*'^Sii(fi})do}y  R22ir)=  j  e‘'^S22((0)da)). 
-Ji  -n 

When  Xi  leads  X2,  Eq.  2.144  gives 


,~iru) 


(2.151) 


(2.152) 


r  =  -00 

Here,  since  /?2i(r’)  is  not  symmetrical  on  r-0,  the  cross  spectrum  521(0)  is  a  complex 
function. 


=  Co2i((y)  +  iQu2i(a>), 


(2.153) 


1  1 

Co2i(<u)=—  y  —(/?2i(r)  +  /?2i(-0}  cos r<u,  (2.154) 

2yt  2 

r  =  -ao 

1  "  I 

Qu2i(o)  =  —  —  {-£2i('’)  +  £2i(-r)}  sinrm.  (2.155) 

ZTT  2 


C02i(<u)  and  Qu2i(<u)  are  the  co-  and  quadrature-spectra  ofXi(r)  and  X2(t)  when 
Xi(t)  leads  X2(t). 

Thus  the  cross  spectrum  is  expressed  by  its  absolute  value  and  argument  as 
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f  •>  ">  11/2 

l52l(itl)l  =  Co2i(w) +  Qu2i(w) 


(2.156) 


and 


Arg  [52i(w)]  =  tan  ^ 


Co2i(a>) 

Qu2i(a>) 


da}\s2\((o)\  =  52i(<uW£o 

=  £[<iZi(w)  dZi^O))]. 

In  this  case  the  coherency  function  y2i(“>)  is  defined  as 


(2.157) 


(2.158) 


y2i(<u)  = 


52i(w) 


cov  jdZ^Co))  dZiio})]^ 


(^ii(<u)^22(to)r'^^  (var(dZi(tu)}  var{i/Z2(£t>)j]^/^ 


(2.159) 


Thus  y2i(.o>)  is  the  correlation  coefficient  of  dZ^(ct>)  and  dZ2(a)).  Here, 


0  S  Iy2i(tti)l  at  all  cu.  (2.160) 

Usually  Iy2i{<y)l  or  is  called  the  coherency  function  and  shows  the  extent 

to  which  X2,  t  and  Xi,  r  are  linearly  related. 

2.62  Linear  Responses 

Suppose  there  is  a  linear  system,  X,  being  the  input,  Y,  the  output  as  in  Fig.  2.28.  If 

00 

r,=  X  8tX,.r,  (2.161) 

r  =  -00 

[g,]  is  called  the  impulse  response  function.  If  this  is  physically  realizable, 

gt  =  0  for  X  <  0. 
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INPUT 


OUTPUT 


LINEAR  SYSTEM 


Rg.  2:28  Linear  system. 

However,  we  had  better  treat  for-  *  —  +  *.  because  sometimes  X,  itself  does 
not  express  the  real  cause  of  y„  and  g,  can  even  be  non-zero  at  r  <  0.  We  assume,  for  the 
impulse  response. 


00 


.00 

(2.162) 

Then 

where 

=  IG(<o)|2  sxxi.o>), 

00 

(2.163) 

Gim)  =  X  St 

(2.164) 

G{oi)  is  called  the  transfer  function  or  frequency  responre  function  of  Y,  to  X,.  For 

00 

syy((o)  to  be  fmite  in  total  power  [  SYy{o))da}  <  oo,  sxx{o>)  must  be  a  bounded  function 


Gio)  and 


00 

I  IG(a>)|-< 


dw  <  00.  Accordingly,  by  Parseval’s  relation, 
Xg?  dt<  oo. 


Using  Eq.  2.161  gives 


58 


ac 


and  therefore 


Rrxir)  =  E  [X:Y,^r]  =  gr  E[X:  X,,r-r] 

r  =-oc 
cc 

=  X  gvExxir-r),  (2.165) 

T  =  -oc 


SYxito)  =  —  X  ^  ‘^'Ryxir) 

r  =  -00 

00  ^  cc 

rs  —  00  r=  —  00 


=  G(w)  sxxio)). 


(2.166) 


Accordingly, 


From  Eq.  2.163 


G(a))  = 


•yjaK<w) 


IG(<w)|2  = 


syy((o) 

sxx(o>) ' 


(2.167) 


(2.163’) 


These  equations  show  that,  from  the  spectra  of  output  and  input,  we  get  the  IG((y)l*,  the 
response  amplitude  fimction,  but  if  we  need  the  complete  response  function  including  the 
phase  relation,  we  have  to  use  the  cross  spectrum  irxftw)  as  shown  in  Eq.  2. 167. 

In  this  ideal  linear  case 


y(<y)^  = 


52i(<y) 


{jll(to)  J22(G>)) 


(2.168) 


The  coherency  ly(m)P  should  be  1. 

2.6J  Linear  Response  in  the  Presence  of  Noise 

When  noise  is  added  to  the  output  T,,  as  in  Fig.  2.29,  where  we  assumed,  is  real¬ 
valued,  zero  mean,  tmcorrelated  with  T,  and  withX,. 
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y;  =  Y,+N,  =  Y^g,X._,  +  N,  , 


Then, 


Fig.  2.29.  Linear  system  in  the  presence  of  noise. 


Rrxir)  =  E[X,*  Y,U 


because, 


Therefore 


E[XrN,^,]  =  0. 


00  00 

srxio})  =  —  X  Z  8tRxAr-r)' 

r  =  —00  f  =  ~oo 

V  J 


=  G{(o)sxxi(o)  =  syx^o)) 


(2.169) 


=  X  gx  E{XtX,^r-x\^E{XrN,^r\ 


=  X  8rExx{r-T), 


(2.171) 


(2.172) 
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G((o)  = 


SYXi<J^) 

Sxx((0) 


(2.173) 


This  shows  that,  if  we  take  the  cross  spectrum  of  T/  withX,,  the  frequency  response 
function  G(tu)  can  be  obtained  as  the  ratio  of  to  the  spectrum  of  input  sxxii^^)  as 
was  the  ideal  case  in  Eq.  2.167,  because  srxfttt)  ts  not  affected  by  the  existence  of  noise 
N,.  However,  taking  the  spectrum  from  Eq.  2.169  gives 


Syria))  =  \Gia})\hxxia})  +  smio))  =  Syria)) -i-  sssio))-  (2.174) 


Therefore,  the  coherency  y^(a>)  is 


yHo))  = 


Sy'xiO)) 

sxxio))  Syria)) 


f  Jyx(a>)|7 
sxx(o>) 


(G(m)p{^rr(Q>)  -  SNsico)} 

^  irrio)) 


SNNia^) 

Syria)) 


l-V^io)). 


(2.175) 


Here 


Vio))^  = 


SNNiO)) 

Syria)) 


(2.176) 


is  called  the  residual  error. 

These  results  show  that  it  is  essential  to  calculate  the  cross  spectrum  to  get  a  good 
estimate  of  the  frequency  response  function. 

2.6.4  Multiple  Inputs  Multiple  Outputs  Case 

More  generally  for  k  multiple  inputs  and  /  multiple  outputs,  as  in  Fig.  2.30,  the 
output  is  expressed  by 
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00 


X  5'1.t  ■X,-,-T+  X  ^'2.r^2  /-r+  •  •  ■  •  X  Sik.r  ^k  i-r  > 


T  =-00 


T  =  -oo 


T  =  -00 


+  •  •  •  •  X  Sik.r  Xk.,-z,  /  =  1  to  /.  (2.177) 

T  =  -0D 


or  in  vector  form. 


00 

y,=  X  s,x,.r. 

r  =  -00 

[/  X  1]  (IX  i]  [ir  X  1] 


(2.178) 


k 


SYSTEM 


Fig.  2.30.  Multiple  inputs  multiple  outputs  system. 

If  the  output  is  expressed  by  spectral  representation,  then 

7t 

Yij  =  je^  d2^\w)  /  =  1  to  /,  (2.179) 


where 


=  G,.  i(co)d2^^\(o)+  ....  +G,.*(cy)dZi%)  /  =  1  to  /.  (2.180) 


2.6J  Multiple  Inputs  Single  Output  Case,  Multiple  and  Partial  Coherencies 

A  multiple  inputs,  single  output  case,  as  in  Fig.  2.31,  is  a  special  case  of  Eq.  2.177. 
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Fig.  2.31.  Multiple  inputs  single  output  system. 


^t=  X  +  X  ■^2,/-t  +••••+  X  ^kj-f 


r  =  -oc 


r  =  -00 


Thus 

k  k  a>  00 

RYYir)=Y  XXX  8i,a  sip  Rij  (ez-^-^r) 

1=1  ;  =  1  a  =-00  ^=-00 


and 


t  it 


5yi<C0)  =  X  E 

.=1 y=i 


In  the  same  way, 

it 

SYfw)  =  X  G'KgJ),  5j/C0), 

i=l 


where  =  RxpCjir),  s./tu)  =  =  ^ix/o>)- 

The  input  is 


Xt  —  [.^1,1  ,^2  /  ....  Xfc  J. 


Then  setting 


G(<o)=  [Gi(iy),G2(a>)  .  .  .  G*(g>)], 


(2.181) 


(2.182) 


(2.183) 


(2.184) 


(2.185) 


(2.186) 
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sn((o)  si2(a))  . 

S2l(Of)  S22U0) 

S2Jc(0}) 

sxxico)  = 

/ici  siaiio)  . 

■  Skkioj). 

•Sll(to)  5i2(tt>)  . 

G\(fo) 

J2i(Q>)  • 

S2i{0>) 

syy{(o)  =  [Gi{a})G2i(o)  .  .  .  G*(co)] 

• 

gives 

Sk\{(0)  Sk2i<0)  . 

.Gk(wX 

syrico)  =  G'(<o)  sxx{(o)  G* 

Here  G  '{(o)  is  the  transpose  of  the  vector  G(a)). 

(W). 

Also 


‘syiico)' 

'su((0)  Si2((0)  ....  SiJ^O)) 

'GiicoY 

Sy2 

S2l((0)  S22(Q>)  S2Jt((0) 

M  Sjaico)  ....  Ski(Q}), 

Gk((o\ 

gives 


Therefore 


G(Q))=  Sj^(Qf)  ■  SYx(of), 


or 


(2.187) 


(2.188) 


(2.188  ’) 


(2.189) 


(2.189’) 


(2.190) 
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‘Gi((oy 

‘syi((0)' 

Gzio)) 

1 

Sy2i(0) 

sxx<.a)) 

GkiO)), 

SYkiO)), 

Thus  the  multiple  coherency  is 


yh(^)  = 


syx(co) 

sxx(0)) 


srr(o>) 

sxx((o) 


Syx(0))  SyxCi^)  1  ^  s 

= - — - —  = - —  G  (co),  SYx(o)).  (2.191) 

Sxxioi)  Syria))  syrico) 

Here,  as  mentioned  by  L.  Tick,^^  H.  Akaike,^^  and  Enochsen,^  the  conditional 
spectrum  ■Sj£|(<u)  is  defined  as 


^i£i(co)  =  5ii(<d)-522(w)  X 


^12(0^) 

S22(a)) 


=  Su((l))- 


lsi2(a))l^ 

S22(a)) 


1- 


l■yl2(Q^)l^ 

^ii(<y),522(ty) 


=  Jii(ca){l-yi2(a>)j. 


(2.192) 


This  shows  the  spectrum  ofXjX]  under  the  condition  that  X2  has  occurred,  that  is,  the 
spectrum  of  XiXi  masked  the  effect  of  X2. 

Generally,  if  the  conditional  spectrum  of  Srx<a>).  under  the  condition  that  xj  has 
occurred,  is  expressed  by  Syj  ^(<u),  then  the  partial  coherency  is  defined  as 


^17.12...)...*^^^“ 


(2.193) 
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CHAPTER  3 


CONSIDERATIONS  ON  THE  IMPROVEMENT  OF  COHERENCY  FUNCTIONS 

3.1  INTRODUCTION 

Coherency  functions  y((o)  can  be  considered  good  clues  to  fmding  the  extent  to 
which  the  system  can  be  approximated  as  linear  to  the  input  X,  which  can  also  be  consid¬ 
ered  linear.  If  there  is  noise  in  the  output  Y,,  as  was  discussed  in  Section  2.6.3,  the 
coherency  function  will  be  reduced,  usually  to  coherencies  of  less  than  1.  It  is  difficult  to 
find  the  real  reason  for  this  reduction,  although  Eqs.  2.175  and  2.176  tell  us  it  is  the  effect 
of  noise.  Here  not  only  noise  in  the  output  but  also  computational  error,  statistical  bias, 
nonlinearity  of  the  response  characteristics,  and  the  effect  of  feedback  are  all  counted  as 
noise  in  the  result.  All  the  effects  that  cannot  be  accounted  for  by  linear,  open  relations 
are  counted  as  noise.  The  author  has  made  a  few  suggestions  for  improving  the  nonpara- 
metric  spectrum  analysis  and  the  resulting  computation  of  coherencies. 

3.2  SHIFT  OF  THE  OUTPUT  IN  CALCULATING  THE  CROSS  SPECTRUM 

Rii(r),  the  auto  correlation  function  of  a  real  process  i(r),  is  an  even  function  and 

shows  a  maximum  value  at  r  =  0,  the  origin  of  the  lag  r.  The  lag  window  applied  to  this 
auto  correlation  to  get  consistent  estimates  of  the  spectrum  ordinates  is  also  usually  an 
even  function  and  has  its  peak  at  r  =  0,  and>v(0)  =  1,  as  in  Fig  2.11.  Accordingly, /f(0) 
remains  unchanged  and  then  gradually  decreases  toward  r-±  M.  This  procedure  keeps 
the  most  reliable  and  important  part  of  the  correlation  near  r  =  0  almost  unchanged  and 
reduces  the  contribution  of  the  correlation  at  larger  values  of  r  near  ±  M,  where  the 
correlation  is  less  reliable,  to  nearly  zero,  preventing  the  formation  of  large  negative  lobes 
in  the  spectral  window.  This  way  is  reasonable  to  get  consistent  estimates  of  the  spectral 
ordinates  and  make  the  spectral  window  fulfill  conditions  1-5  in  Section  2.5.4. 

However,  in  getting  the  cross  correlation,  if  we  use  lag  windows  in  the  same  way, 
sometimes  important  information  is  lost,  and  the  result  is  an  apparent  reduction  in  coher¬ 
encies.  This  was  studied  theoretically  by  N.  Akaike  and  Y.  Yamanouchi^’  (1962),  but  a 
more  intuitive  explanation  by  this  author  [Yamanouchi^  (1961)]  is  given  here. 

In  cross  correlation  Rj/r),  if  the  phase  relation  of  the  output  J{t)  lags  or  leads  the 
input  j(f)  considerably,  the  maximum  value  of  the  cross  correlation,  which  is  not  the  even 
function,  will  no  longer  lie  at  the  origin  r  =  0  but  will  lie  at  a  larger  or  minus  value  of  r 
(distant  from  the  origin).  In  applying  lag  windows,  if  we  set  the  maximum  of  the  lag  win¬ 
dow  at  r  =  0,  sometimes  important  information  is  lost  at  the  peak  of  cross  correlation.  A 
small  value  of  the  lag  window  will  be  multiplied  to  the  peak  value  of  the  cross  correlation 
at  a  larger  or  minus  value  of  r,  and  will  keep  the  less  important  part  of  the  cross  correla¬ 
tion  almost  unchanged  by  the  windows,  as  shown  in  the  example  in  Fig.  3.1.  This  not 
reasonable,  so  the  author  suggested  shifting  the  origin  of  the  cross  correlation  to  its  pe-tk 
point  and  then  applying  the  lag  windows.  The  amount  of  the  change  of  phase  from  this 
shift  ro  "should,  of  course,  be  used  later  to  modify  the  phase  relation  by  rooj. 
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V  ^  \/ 


u  V  V 


r-r. 


Fig.  3.1 .  Shift  of  cross  correlation  in  applying  the  lag  window. 

This  alignment  technique  can  be  expressed  as  a  shift  in  lag  windows  as  well  as  a 
shift  in  the  origin  of  the  correlation,  or  as  a  shift  of  the  output  time  series  relative  to  the 
input  time  series. 


,  j  (W-l)-ro 

Sj,i(o)  =  —  wir-ro)Rj,ir)e-^'',  (3.1) 

^  r=-(iV-l)-ro 

where  w(r)  is  the  standard  type  of  lag  window  for  estimation  of  the  auto-spectra. 

Thus 


1 

Sj,<(0)  =  —  X  wir)Rj,ir  +  (3.2) 

^  r=-(N-l) 

So  if  we  set 
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Sji{(D)  =  -^  X  (3 ■  3 ) 

then 

4(a>)  =  Sj,i(o)  X  (3.4) 

The  phase  is  shifted  by  <t>ro  by  this  shift  in  the  origin.  Accordingly  the  phase  of  the 
output  should  be  modified  by  -  roti>  after  the  cross  specffum  analysis.  Figure  3.2  shows  an 
example  of  this  shift  in  the  analysis  of  a  5-ft  model  ship  rolling  in  tank  waves (beam 
sea,  without  advance  speed).  From  this  figure,  we  find  the  amount  of  shift  on  cross  corre¬ 
lation  should  be  ro  =  9.  The  results  of  the  analysis  are  shown  in  Figs.  3.3  and  3.4.  The 
improvement  in  the  coherencies  due  to  the  shift  is  clearly  shown  in  the  upper  part  of  Fig. 
3.4.  The  improvements  in  phase  relation  and  in  gain  are  also  apparent,  although  these  im¬ 
provements  are  partly  due  to  the  large  m{=  60  >  40). 


Fig.  3.2.  Auto  and  cross  correlations  of  wave,  roll,  roll-wave 
for  a  model  ship  in  tank  (run  834). 

(From  Yamanouchi.^’) 

Of  special  interest  to  us  is  that  the  improvement  in  coherencies  is  also  reflected  in 
the  change  of  bandwidth  of  the  cross  spectrum.  Before  the  shift  in  origin,  the  co-  and 
quad-spectra  in  Fig.  3.3  had  a  narrower  bandwidth  than  after  the  shif'  showing  that  the 
blurring  effect  of  the  spectral  window  is  smaller  for  the  cross  spectra  with  broader  band¬ 
width  than  for  those  with  narrower  bandwidth.  Thus  this  shift  improved  the  estimate  of 
the  cross  spectra  by  reducing  the  leakage  of  power.  Although  we  refer  to  improvement, 
we  do  not  have  experimental  data  in  regular  waves  for  comparison  in  this  rua  However, 
in  Figs.  3  . 5-3 .7,  which  show  the  same  kind  of  result  for  run  832  and  the  data  in  regular 
waves  (Fig.  3.7),  we  notice  the  same  tendency  and  conclude  that  the  results  obtained  by 
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this  shift  are  best  in  this  analysis  and  closest  to  the  experimental  data  obtained  in  regular 
waves. 


CROSS  SPECTRUM 


Rg.  3.3.  Auto  and  cross  spectra  of  wave,  roll,  roll-wave  (run  834)  (max. 
lag  no.  /77s60  with  shift  and  msAO  without  shift). 

(From  Yamanouchi.®’) 

3.3  IMPULSE  RESPONSE  FUNCTIONS  FROM  THE  CROSS 
AND  AUTO  CORRELAnONS 

By  Eq.  2.169  for  a  linear  system, 

00 

y,=  X  8uX,^  +  N„  (3.5) 

M  =  -oe 

where  gu  is  the  linear  impulse  response  function  of  the  linear  system,  X,  and  Y,  are  the 
input  and  output,  respectively,  and  N,  is  the  noise,  uncorrelated  with  X,  and  Y,.  By  Eq. 
2.164 
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(3.6) 


OC 

G(o;)=  X 

u  =-oe 

where  G(a>)  is  the  frequency  response  function  of  this  system.  Then  as  already  shown  by 
Eqs.  2.165, 2.170;  2.166,  2.172, 

X  Rxx^r-u)  (3.7) 


Fig.  3.4.  Frequency  responses  of  roll-wave  (run  834) 
(r77=60  with  shift  and  /n=40  without  shift). 

(From  Yamanouchi.3’) 


STodp})  =  G((a)  sx]({o>)  (3-8) 

If  we  assume  that  (with  -  »  <u<  <» )  can  be  approximated  by  finite  terms  of 
gu  (with  -n<u<n),  then  Eq.  3.7  gives  the  matrix  of  Eq.  3.9, 
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CORREUOGRAM 


x10~*  (REET-DEGREE)  »10~* 


T 


Fig.  3.5.  Auto  and  cross  correlation  of  wave,  roll,  roll-wave  of  a 
model  ship  (run  832). 

(From  Yamanouchi.3i) 


^xy(0)  ^xx(l) . Rxxin)  ....  Rxx{2n) 

Rxxi^)  RxxiO) . Rxxin-1)  .  .  Rxxi2n-l) 


Rxxin)  •  •  •/fxx(0)  ....  l^xxin) 


Rxx{2n-l)R^2n-2)  .  .  Rxxin-l)  .  . 
Rxxi2n)  R^2n-\)  .  .  RxxHn)  ....  ^xx(0) 


using  the  relation /?(-  r)  =  R(r).  Here  for  the  cross  correlations,  we  use  the  shifted  version 

of  Ryx^r),  as  Ryx(0)  is  to  be  the  maxima  over  the  range  r  =  -n  to  n,  and  for  the  auto  cor¬ 
relation,  the  range  0  to  2n.  The  matrix  on  the  right  hand  side  of  Eq.  3.9  is  large  on  the 
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Fig.  3.6.  Auto  and  cross  spectra  of  wave,  roll,  roll-wave  (run  832) 
(m=60  with  shift,  and  m=40  and  120  without  shift). 

(From  Yamanouchi.^’) 
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diagonal  and  is  symmetrical  around  the  diagonal.  The  solution  of  Eq.  3.9  gives  gu  over 

the  range  u  =  -  n  to  n.  This  author  [Yamanouchi^^]  believes  this  method  provides  the  lag 
window  free  estimate  of  the  impulse  response  function,  or  the  spectral  window  free  esti¬ 
mate  of  the  frequency  response  function  from  Eq.  3.6.  The  choice  of  the  lag  window  or 
the  spectrum  window  is  a  serious  problem  in  getting  a  good  estimation  of  the  spectra  and 
the  frequency  response  relations  of  the  output  to  the  input,  as  was  discussed  in  Sections 
2.5.2-2.5.6.  This  method  eliminates  the  problem  of  windows  and  frees  the  calculation 
from  their  blurring  effect.  The  use  of  Eq.  3.9  might  eliminate  also  some  of  the  uncertain¬ 
ties  and  errors  in  the  algorithm  for  the  correlation. 

Figures  3.8-3.11  show  the  analysis  of  the  rolling  of  model  ships  afloat  in  irregular 
beam  seas.  g„  for  -  90  to  90  was  analyzed  using  /?ra(a;)  of  -  90  to  90  and  Rxx  of  0  to 
180.  The  correlations  in  Fig.  3.8,  where  the  cross  correlations  are  already  shifted  by  9, 
are  shown  in  normalized  form  and  the  calculated are  shown  at  the  top  of  Fig.  3.9  for 
«  of  -  30  to  -t-  30  only,  though  the  g„  were  computed  for  «  of  -  90  to  -t-  90.  For  compari¬ 
son,  the  impulse  response  function  obtained  as  the  Fourier  inverse  transform  of  the 
frequency  resptonse  function  calculated  by  cross  and  auto  spectrum  analysis  is  shown  at 
the  bottom  of  Fig.  3.9.  They  look  very  similar. 

The  frequency  response  function  obtained  as  the  Fourier  transform  of  the  impulse 
response  function g^  calculated  by  this  method  is  shown  in  Fig.  3.10  together  with  the 
results  of  cross  and  auto  spectral  analysis.  Again,  the  frequency  response  function 
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Fig.  3.7.  Frequency  responses  of  roll-waves  (run  832). 

(From  Yai^anouchi.®’) 

IG(a>)l,  i.e.,  the  amplitude  gain,  from  these  different  methods  is  very  similar.  From  the 
time  history  of  inputs  X,  of  the  waves,  and  the  use  of  Eq.  3.5  which  neglects  the  noise 
N,,  the  output  Y,  was  synthesized,  using  values  of  u  =  -  90  to  90,  as  shown  in  Fig. 

3.11.  These  results  show  that  this  method  gives  reasonably  good  results. 

This  example  is  for  the  roll  of  a  model  ship  where,  because  of  the  smaU  damping  of 
the  motion,  the  impulse  response  function  g„  is  slow  to  decay.  We  need  a  larger  number  of 
terms,  i.e.,  gu  for  a  longer  range  of  u.  With  much  larger  damping,  as  with  the  heave  or 
pitch  response  of  a  real  ship,  this  author  believes  we  will  get  better  results. 

Here  the  more  statistical,  stricter  estimation  of  reliability  or  confidence  in  the  results 
is  lacking.  This  author  found  later  that  the  procedure  was  the  same  as  that  to  solve  the 
Yule-Walker  equations  used  for  the  AR-model  fitting,  that  will  be  mentioned  in  Part  II. 
Accordingly,  the  above  menboned  belief  was  more  strictly  examined  stabsbcally  as  the 
choice  of  order  in  AR-model  fitting,  as  menboned  in  Secbon  S.S. 

3.4  EXAIVDPLE  OF  MULTIPLE  INPUT  ANALYSIS,  A  TRIAL  FOR 
NONLINEAR  ANALYSIS  OF  SHIP’S  RESPONSE 

Here  a  mulbple  input  analysis  of  a  ship’s  behavior  at  sea,  performed  by  this  author 
[Yamanouchi^^],  will  be  shown  to  demonstrate  the  usefulness  of  the  method.  Often,  one 


Rg.  3.8.  Auto  correlation  of  waves,  roll,  and  cross  correlation  of 
wave-roll  of  a  model  ship  (shift  r©  =  9). 


Rg.  3.9.  Impulse  response  of  roll  to  wave  height  (without  advance  speed). 

(From  Yamanouchi.32) 

output  is  analyzed  as  the  response  to  a  single  definite  input.  This  approach  is  reasonable 
when  the  real  source  of  excitation  for  the  system  is  actually  single  and  no  other  source 
need  be  considered,  even  if  several  outputs  are  correlated  or  combined.  For  example, 
even  though  the  sway,  roll  and  yaw,  or  pitch,  heave  and  surge  motions  of  a  ship  are 
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UNDEHSTANOING  AND  PREDICTION  OF  SHIP  MOTIONS 


RELATIVE  ERROR 


Rg.  3.10.  Frequency  response  of  roll  to  wave  height. 
(From  Yamanouchi.®^) 


I 


Fig.  3.11.  Comparison  of  synthesized  and  observed  output  of  roll 
(From  Yamanouchi.32) 


[ 

[ 


correlated  and  combined  with  each  other,  we  can  get  the  response  of  a  single  roll,  sway, 
or  yaw  and  so  on,  to  the  wave  (either  slope  or  height),  as  frequency  resp>onse  functions  by 
solving  the  simultaneous  equations  of  motion  of  the  necessary  degree  of  freedom,  consid¬ 
ering  the  waves  as  only  one  source  of  excitation. 

This  is  not  the  case,  however,  when  one  output  must  be  considered  as  the  response 
to  many  related  inputs.  Shown  here  is  an  example  that  this  author  encountered  in  the 
analysis  of  a  ship’s  motion  and  stress.  The  ship  was  a  cargo  liner  on  the  New  York  line, 
under  service  in  winter  on  the  North  Pacific  Ocean.  ^  The  relative  wave  heights  of  en- 
coimteiing  waves  were  measured  by  an  on-board  ultrasonic  sensor  located  on  the  side  of 
the  hull  near  midship.  Because  of  the  poor  position  of  the  measuring  device,  no  attempt 
was  made  to  convert  the  readings  to  absolute  encountering  wave  heights.  However,  the 
real  encountering  waves  are  only  one  input  to  the  stresses  and  the  motions  of  the  shir>.  In 
analyzing  the  transverse  stresses  induced  on  the  web  frame  near  midship,  this  author  tried 
to  express  the  effect  of  real  waves  by  the  relative  wave  heights  measured  at  the  side  of  the 
ship  and  by  supplementing  with  other  input.s  like  rolling,  pitching  and  vertical  accelera¬ 
tion  measured  at  the  same  time.  These  additional  inputs  were  correlated  with  each  other, 
as  shown  in  Fig.  3.12,  and  multiple  input  analysis  techniques  were  adopted  to  analyze  the 
response. 


Fig.  3.12.  Multiple  inputs  interpretation. 

(From  Yamanouchi.^) 

With  stress  as  the  output  and  such  inputs  as  relative  wave  height,  rolling,  pitching, 
and  vertical  acceleration,  and  with  various  numbers  of  inputs,  the  effects  of  some  particu¬ 
lar  inputs  were  investigated  by  checking  the  multiple  and  partial  coherencies  that  show 
the  effects  of  a  particular  input  in  the  presence  of  one  or  more  other  inputs. 

Another  trial  investigation  was  the  analysis  of  nonlinearity  of  output  to  input.  Con¬ 
siderations  on  nonlinearity  that  provide  another  reason  for  getting  a  poor  coherency 
function  are  generally  treated  in  Part  HI  of  this  lecture,  but  for  convenience  one  trial  is 
described  here  in  relation  to  multiple  input  analysis.  Figure  3.13  is  an  example  of  a  record 
of  simultaneous  measurement  of  many  responses  of  this  ship  such  as  roll,  relative  wave 
height,  vertical  accelerations  at  four  points,  stress  on  the  web  frame,  revolution  and 
torque  of  the  propeller,  helm  angle,  yaw,  encounter  time  of  waves  to  the  stem,  and  so  on. 
Some  of  the  responses  were  picked  up,  sampled,  digitized,  and  punched  on  a  tape  after 
the  test.  (At  that  time  data  were  processed  using  punched  tapes.) 
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RECORD  ON  VISI-COROER  (t) 
j|'fTri"i  i»^i  T  rr3X»  rmi" 


Fig.  3.1 3.  Examples  of  record  on  visi-corder  ((es(  no.  220). 


Table  3.1  summarizes  the  measurements  of  the  responses  and  shows  that  these 
thirteen  representative  runs  cover  a  good  variety  of  sea  conditions,  wave  heights,  and 
directions.  This  is  reflected  in  the  variation  of  normalized  correlations  and  spectra,  for 
example  in  the  rolling,  pitching,  stress,  and  relative  wave  heights  in  Fig.  3.14  and 
Fig.  3.15. 

If  we  assume  a  variety  of  environmental  conditions,  it  is  possible  that  various  sea 
conditions  might  appear  ideally  by  the  same  chance,  and  the  average  of  all  normalized 
wave  spectra  might  give  us  a  nearly  white  spectrum,  or  the  spectrum  of  while  noise.  Then 
the  averaged  response  might  show  the  characteristics  of  the  response  to  the  while  noise, 
i.e.,  the  characteristics  of  the  response  functions  themselves,  as  was  discussed  by 
Yamanouchi,  et  al.^^  The  average  normalized  correlation  diagram  or  correlogram  and 
averaged  spectra  are  shown  in  Fig.  3.16. 


These  averaged  rolled  and  pitch  spectra  are  not  the  ideal  ones,  but  quite  reasonably 
show  the  smooth  peak  at  its  natural  frequency.  Moreover,  the  correlation  functions  of 
roll,  pitch  and  stress  show  the  beautiful  forms  of  damped  oscillation.  The  averaged  spec¬ 
trum  of  stress  has  a  smooth  peak  at  twice  the  frequency  of  the  roll  natural  frequency.  The 
natural  frequency  of  the  stress  response  should  be  in  a  higher  frequency  range,  but  must 
have  been  cut  off  by  the  filters  and  should  not  appear  in  the  averaged  spectra.  The  peak 
that  does  appear  at  the  double  frequency  of  the  roll  natural  frequency  might  indicate  that 
the  transverse  stress  induced  at  the  web  frame  is  quadratic  nonlinear  to  the  rolling  motion. 


To  check  this  possibility,  an  artificial  process  of  rolling  square  was  made  as  shown 
in  Fig.  3.17.  TTiis  naturally  has  a  biased  mean  as  shown  in  its  variational  form  in  the  time 
series  and  also  in  the  spectrum  in  Fig.  3.18,  where  the  power  value  near  o  =  0  is  large.  In 
the  next  step,  a  digital  high  pass  filter,  shown  by  £q.  2.132,  was  applied  to  this  roll 


squared  process  as  X„(f)  =  The  filtered  roll  squared  process  appears  at  the 


bottom  of  Fig.  3.17  and  its  spectrum  in  Fig.  3.18.  In  Fig.  3.18,  we  find  the  shape  of  the 
spectrum  of  the  filtered  roll  squared  process  is  similar  to  that  of  the  averaged  spectrum 
of  the  stress,  which  validates  the  assumption  that  the  stress  is  a  quadratic  and  nonlinear 
response  to  rolling  motion. 


The  lower  two  graphs  in  Fig.  3,19  show  the  results  of  single  input-output  spectrum 
analysis,  with  the  stress  as  output  and  relative  wave  height  or  rolling  as  a  single  input 
Coherencies  values  are  so  low  that  the  stress  cannot  be  the  output  of  only  the  relative 

wave  height  nor  only  of  rolling.  The  top  of  Fig.  3.19  shows  on  the  contrary  that  relative 
wave  height  is  fairly  well  explained  just  by  rolling;  the  coherency,  especially  in  the 
frequency  range  in  which  rolling  has  reasonable  power,  has  a  value  pretty  near  1 . 


Since  we  found  that  the  stress  was  quite  possibly  quadratic  and  nonlinear  to  the 
roiling,  the  next  step  was  to  find  the  response  of  stress  to  the  single  roll  squared  process, 
(roll)^.  To  eliminate  the  effect  of  large  bias  on  (roll)^,  the  response  of  stress  to  the  single 
(roll)^  filtered  process  was  also  obtained.  The  results  are  shown  in  Fig.  3.20,  and  the 
coherency  is  again  rather  low. 


To  check  the  coexistence  of  many  other  inputs,  multiple  input  analysis  was  intro¬ 
duced.  Stress  was  considered  as  the  output  of  rolling,  pitching,  relative  wave  height,  and 
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Table  3.1 .  Summary  of  seaways  and  particulars  of  measurement 
and  analysis  for  test  runs. 

(From  Yamanouchi.®®) 
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Fig.  3.16.  Averaged  correlograms  and  spectra  (normalized). 
(From  Yamanouchi.“) 
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(From  Yamanouchi.®®) 
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T  NO  2D7 


Rg.  3.18.  Spectra  of  roll.  {roll)^.  (roloMiltered. 

(From  Yamanouchi.^) 

heaving.  Heaving  itself  was  not  measured,  but  its  effect  was  assumed  to  be  included  in 
the  vertical  acceleration  of  a  representative  point  and  also  to  be  combined  in  pitching, 
rolling,  and  relative  wave  height.  Examples  of  four-input  analysis  are  shown  in  Figs.  3.21 
and  3.22,  in  which  stress  as  the  output  is  given  on  channel  S  and  the  inputs  of  rolling  on 
channel  1,  pitching  on  channel  2,  relative  wave  height  on  channel  3,  and  (roll)*  filtered  on 
channel  4.  Figure  3.21  shows  five  auto  conelations  of  ou^ut  and  input  R55,  Rn,  R22.  R33. 
R44;  ten  cross  conelations  of  combinations  of  the  one  output  with  four  inputs  R51,  Rs2. 
R53.  R54;  R21;  R41.  R42.  R43;  R31.  R32;  and  five  auto  spectra  of  stress  S55,  rolling  Sn,  pitch¬ 
ing  S22.  relative  wave  height  S33,  and  roll  squared.  Figure  3.22  shows  conditional  (partial) 

gains  and  phase  relations  as  ^sw-rpr^^  and  ^ sp}~~rfw  in  ihe  form 

of  absolute  values  and  arguments.  The  notation  follows  the  convention  already  explained 
in  Section  2.6.5  on  partial  coherencies.  For  example. 


f^SP-RWR^^O))  - 


Ssp-RWR^(<^) 

Spp^WR^io>) 


shows  the  frequency  response  of  stress  to  R:  rolling,  W:  relative  wave  height,  and  R^: 
squared  roll  filtered  processes  under  the  condition  that  the  effect  of  pitch  has  been 
masked.  In  the  same  way,  multiple  and  partial  cross  spectral  analysis  was  tried  with  stress 
as  one  output  (4),  and  roll  R  (1),  pitch  P  (2),  and  (roU)^  filtered  R^  (3)  as  three  inputs.  In 
another  one  output/three  input  cases,  pitching  P  was  replaced  by  vertical  acceleration  A. 
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Pig.  3.19.  Coherencies  for  single  input-output  relations. 
(From  Yamanouchi.33) 


The  results  of  spectrum  analysis  of  the  first  three-input  case  are  shown  in  Fig.  3.23  and 
Fig.  3.24. 

Auto  correlations  R44,  R33,  R22.  Ru.  and  all  combinations  of  cross  correlation  R41, 
R42,  R43,  R31,  R32.  R21  and  auto  sp-^*  tra  S44,  Su.  S33  are  shown  in  Fig.  3.23.  Partial  gains 

and  phase  relations  and  _pp  are  shown  in  Fig.  3.24.  The  effects 

of  the  increase  in  probable  inputs  are  more  clearly  shown  in  the  multiple  coherencies,  and 
the  effects  of  several  inputs  when  the  effect  of  a  certain  input  was  masked  are  shown  on 


1.0 

0.8 

0.6 

0.4 

02 

0.0 


COHERENCIES 
TEST  NO.  207 


:  STRy(ROLL)2 


OMEGA 


COHERENCIES 
TEST  NO.  207 


'  STRy(ROLL)2-<itt. 


Fig.  3.20.  Coherency  of  squared  rolling  process  and  filtered  squared 
rolling  process  to  the  stress. 

(From  Yamanouchi.33) 

the  partial  coherencies.  The  results  for  two-input,  three-input  and  four-input  cases  are 
compared  in  Fig.  3.25. 

This  figure  indicates  that  the  multiple  coherency  shows  higher  levels  as  more  inputs 
are  taken  into  account.  Moreover,  examining  the  partial  coherencies, 

VsR^-RPW  others,  at  the  bottom  figure  of  Fig.  3.25,  we  find  that  the  contributions  of 
rolling  and  of  (roll)^-filters  are  large  compared  with  the  other  inputs.  This  finding  sup¬ 
ports  the  assumption  of  quadratic  nonlinearity  of  transverse  stress  to  the  rolling  motion. 

Figure  3.26  shows  the  multiple  and  partial  coherencies  for  the  two-inputs  and 
three-inputs  cases,  and  here  again  we  find  the  coherency  improves  with  increasing  num¬ 
ber  of  inputs,  including  the  (roll)^-filtered  process.  Also  the  contributions  of  rolling  and 
(roll)^-filtered  are  large,  compared  with  the  contribution  of  pitching.  The  multiple  coher¬ 
ency  is  very  close  to  1  around  the  frequency  range  in  which  the  output,  here  the  stress, 
has  important  response  to  input. 
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(From  Yamanouchi.' 


T.NO.  2J4 
315®  {kg/nm*;* 


Fig.  3.24.  Example  of  three-inputs  analysis;  amplitude  gain  and  phase  shift. 

(From  Yamanouchi.®®) 

To  show  more  clearly  the  usefulness  of  multi  input  analysis,  apart  from  the  nonlin¬ 
earity  of  responses,  another  example  used  the  vertical  acceleration  as  a  linear  output  and 
rolling,  pitching,  and  relative  wave  height  as  inputs.  The  multiple  and  partial  coherencies 
are  given  in  Fig.  3.27,  in  which  the  multiple  coherency  is  almost  1  at  the  important  range 
of  frequencies  and  the  contribution  of  rolling  is  large  for  »his  vertical  acceleration,  as  we 
expected. 

Figure  3.28  is  another  example  of  two-input  and  three-input  analysis  for  stress 
which  does  not  take  into  account  the  nonlinearity  of  rolling.  Comparison  with  Figs.  3.25 
and  3.26  at  the  important  range  of  frequencies  for  stress  shows  lower  coherencies  when 
the  nonlinear  response  of  rolling  is  not  taken  into  account. 

Later,  the  author  found  that  taking  the  input  process  squared  as  an  input  is  very  rea¬ 
sonable  in  discussing  the  quadratic  response  character  of  the  output  as  shown  in  Section 
11.4,  in  Part  in. 
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Fig.  3.25.  Examples  of  multiple  and  partial  coherencies  -  (I);  stress-roll,  pitch,  relative 
wave  height,  and  (roll)^;  comparison  of  two-,  three-,  and  four-inputs  cases. 

(From  Yamanouchi.^*^) 
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Fig.  3.26.  Examples  of  multiple  and  partial  coherencies  -  (II);  stress-roll, 
pitch,  and  (roll)^;  comparison  of  two-  and  three-inputs  cases. 

(From  Yamanouchi.^) 


0.5  1.0  1.5  2.0  OMEGA 


Rg.  3,27.  Example  of  multiple  and  partial  coherencies  -  (111);  vertical  acceleration- 
roll,  pitch,  and  relative  wave  height:  comparison  of  two-  an 
three-inputs  cases. 

(From  Yamanouchi.®®) 
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Fig.  3.28.  Example  of  multiple  and  partial  coherencies  -  (IV);  stress-roll,  vertical 
acceleration  and  relative  wave  height;  comparison  of  two-  and  three- 
inputs  cases,  when  the  (roll)^  is  not  taken  into  consideration. 

(From  Yamanouchi.®^) 
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CHAPTER  4 
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CONCLUSION  FOR  PART  I 

In  Part  I,  the  nonparametric  procedure  for  analyzing  the  irregular  time  series  was 
discussed.  At  the  beginning  of  Chapter  2  the  basic  procedure  for  obtaining  the  spectrum 
through  periodograms  was  summarized  and  then  a  few  considerations  for  improving  this 
method  were  introduced  in  Chapter  3.  The  conclusions  derived  from  the  discussion  are  as 
follows: 

1.  The  specffal  analysis  technique  with  the  use  of  periodograms  is  well  established 
and  affords  us  a  powerful  approach  for  analyzing  irregular  phenomena. 

2.  In  sample  compuutions  of  the  correlation  function  as  well  as  of  the  spectra, 
however,  care  is  necessary  in  sampling  the  data.  The  sampling  time  interval,  the  length 
of  record,  maximum  length  of  lag,  and  proper  windows  must  be  chosen  to  get  consistent 
estimates  and  avoid  aliasing,  blurring  by  the  windows,  or  loss  of  reliability. 

3.  Spectral-lag  window  pairs  have  been  proposed  by  many  scholars,  and  were 
shown  that  we  must  be  careful  of  the  effect  of  windows  on  the  reliability,  variability,  and 
resolution  of  the  results. 

4.  Computation  of  the  spectrum  through  paiodograms  by  use  of  the  Finite  Fourier 
Transform  is  the  same  as  that  computed  through  the  correlation  function. 

5.  The  Fast  Fourier  Transform  (FJ.T.)  method  is  a  convenient  algorithm  for  reduc¬ 
ing  the  number  of  operations  in  the  computation,  but  the  considerations  of  the  windows 
are  also  necessary  and  important  in  applying  this  method. 

6.  The  correlation  functions  can  also  be  conveniently  calculated  by  the  F.F.T. 
method  if  proper  precautions  are  taken. 

7.  In  connection  with  the  choice  of  windows,  the  use  of  filters  before  applying  the 
window  is  worth  considering. 

8.  Not  only  the  spectrum  functions  but  also  the  correlation  functions  (correlo- 
grams)  should  be  investigated  carefully  in  estimating  the  character  of  the  process. 

9.  In  applying  the  spectral  window  in  cross  spectral  analysis  or  in  response  analy  - 
sis,  the  shift  of  the  origin  of  the  cross  correlation  (shift  of  the  output  record)  should  be 
considered  to  minimize  the  leakage  of  power  through  the  use  of  spectral  windows  in 
computing  cross  spectrum. 

10.  Cross  spectral  analysis  is  essential  in  the  analysis  of  the  response  process  of  a 
dynamic  system  to  get  full  information  on  the  frequency  response  functions  of  the  sys¬ 
tem.  Cross  spectral  analysis  is  effective  in  reducing  the  effect  of  noise  that  contaminates 
the  output 

1 1 .  The  coherency  function  is  a  good  index  to  the  extent  to  which  the  response  can 
be  expressed  by  linear  relations.  To  make  the  coherency  function  useful  the  computations 
of  all  spectra  must  be  done  properly  according  to  the  preceding  items  4-10  in  getting  the 
coherencies. 

12.  The  impulse  response  function  may  be  obtained  from  the  cross  and  auto  corre¬ 
lations  of  output  and  input,  without  the  trouble  of  choosing  a  window.  This  procedure 
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was  found  later  to  be  closely  related  to  the  AR-model  fitting  technique  discussed  in  Sec¬ 
tion  5.5. 

13.  In  the  analysis  of  seakeeping  data,  multiple  input  analysis  is  helpful.  Parual  co¬ 
herencies  and  the  multiple  coherency  give  a  good  clue  to  the  extent  of  contributions  from 
each  input. 

14.  In  connection  with  consideration  of  the  filtration  of  the  stochastic  process,  the 
model  fitting  techniques  that  try  to  express  the  stochastic  process  by  a  finite  number  of 
parameters  are  found  to  look  promising  to  supplement  the  nonparametric  method.  The 
model  fitting  technique  or  parametric  method  will  be  reviewed  in  detail  in  Part  H. 
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PARXn 


MODEL  FITTING  TECHNIQUES 
(PARAMETRIC  SPECTRAL  ANALYSIS) 

CHAPTERS 

DISCRETE  MODEL,  MODEL  FITTING, 

CORRELATION  AND  SPECTRUM  FUNCTIONS 

5.1.  INTRODUCTION 

In  Part  I,  the  procedures  for  estimating  the  spectrum  nonparametrically  through 
autovariance  from  a  sample  observation  were  sunamarized  and  discussed.  We  noted  that 
the  statistical  consideration  of  each  step  of  the  computations  are  important  in  getting 
reliable  results,  and  the  author  suggested  several  ways  to  improve  reliability.  In  Part  n. 
another  approach  to  getting  reliable  results  from  a  single  or  a  short  obserx'ation  of  a 
process  will  be  discussed.  The  method  introduced  here  is  called  a  parametric  approach, 
because  some  type  of  model  is  fitted  to  the  sample  observation,  and  then  the  parameters 
of  the  model  are  estimated  statistically.  In  looking  for  the  models,  we  can  use  the  knowl¬ 
edge  that  we  already  have  about  the  process,  such  as  the  degrees  of  freedom,  and  the 
physical  characteristics  of  the  equations  of  motion  that  govern  the  behavior  of  the  system 
or  about  the  inputs.  A  few  methods  which  use  the  parametric  approach,  such  as  the 
maximum  likelihood  method  (MLM)  and  the  maximum  entropy  method  (MEM)  are 
essentially  the  same  with  the  one  mentioned  here,  and  will  be  explained  later. 

In  this  analysis,  the  criteria  for  deciding  the  fitness  of  the  model  are  very  important 
The  method  introduced  here,  called  the  “MAIC  Method,”  was  introduced  by  Dr.  H. 
Akaike  of  Japan  and  provides  a  powerful  guide  in  fmding  the  properly  fitted  model,  prop¬ 
er  from  a  statistical  point  of  view.  In  this  analysis,  the  time  domain  expression  of  this 
process,  the  time  histories  themselves,  and  the  correlation  of  the  processes  play  a  big  role 
as  was  pointed  out  in  Sections  1.3  and  2.S.8  and  in  the  conclusion  to  Part  I. 

This  parametric  approach  is  not  yet  well  known  in  the  field  of  naval  architecture, 
although  a  few  books  by  Priestley,^^  Pandit  and  Wu,^  Box  and  Jenkins,^’  and  others^ 
deal  with  it  in  part  or  in  depth.  For  this  reason  Sections  5.2.1  to  5.2.5  give  several  statisti¬ 
cal  models  of  stationary  time  series  in  some  detail  with  simulated  examples  by  this 
author.  Then  criteria  for  choosing  the  model,  estimating  the  parameters,  and  deriving 
the  spectrum  will  be  given  at  the  end  of  Chapter  5. 

Chapter  6  discusses  the  application  of  this  method  to  a  two-variate  process,  an 
input/output  system,  and  the  usefulness  of  this  method  for  the  analysis  of  a  response 
system  with  feedback  is  shown. 

In  Chapter  7,  examples  of  the  application  of  this  method  to  the  analysis  of  seakeep¬ 
ing  data  and  a  comparison  with  analysis  by  the  nonparametric  method  are  shown. 

Finally,  Chapter  8  provides  conclusions  and  summaries  for  Part  II. 
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5.2.  DISCRETE  PARAMETRIC  MODELS 

There  are  maay  statistical  models  for  expressing  tune  series.  Among  them  the  auto¬ 
regressive  (AR)  model,  the  moving  average  (MA)  model,  and  the  mixed  autoregressive 
moving  average  (ARMA)  model  are  the  most  representative  Imear  models  that  we  en¬ 
counter  in  the  analysis  of  irregular  records  of  observations.  In  Part  U.  only  the  linear 
models  are  introduced.  Some  types  of  nonlinear  models  will  be  referred  to  later  m  Part 
ni.  The  order  of  the  AR,  MA,  and  ARMA  models  mdicates  the  degree  of  simplicity  or 
complexity  of  the  models. 

Here  in  Part  H,  the  discrete  time  process  sampled  from  the  continuous  time  process, 
with  an  interval  At  is  expressed  by  (X,  1  and  its  realization  by  X,.  Except  as  otherwise  in¬ 
dicated,  Ar  is  taken  as  1 .  If  Ar  ^  1 ,  each  function  is  easily  transformed  to  proper  form,  as 
was  already  mentioned  in  Section  2.4.3.  The  explanation  of  the  character  of  the  elemen¬ 
tary  models  is  based  largely  on  the  work  of  Priestly‘S  and  Pandit  and  Wu.s^ 

52.]  Pure  Random  Process 

( Ai )  is  called  a  pure  random  process  if  it  is  the  sequence  of  uncorrelated  random 
variables  that  are  stationary  up  to  order  2,  and  is  written 


The  mean  is 

the  variance  is 


X.  =  €,. 


{X,-n)^^=ar  =  0} 


(5.1) 

(5.2) 

(5.3) 


and  the  covariance  function  is 


0 


R{.r)  =  cov.jx,,  = 

R{r)  is  a  function  of  r  only  and  is  normalized  by  a  ‘  as 

Q(r)  = 


r  ^  Q 
r  =  0 


(5.4) 


r  ^0 

r  =  r 


(S.S'l 


Its  spectrum  is  then 


1  ^  X 

5(01)  =  —  y  R{r)e  -  ‘  -2—-  const,  (for-jr  ^  oi  >  tt).  (5.6) 

231  ^  231 

r  =  00 

In  this  case 

I.  When  this  process  is  also  Gaussian,  then  A,  is  not  only  uncorrelated,  but  also 
A,,  Ar,_i,-  •  •  A^.r,  are  independent  of  each  other. 


2.  The  spectrum  s(ai)  is  flat  for  the  interval  of  frequency  tw  for  -  jr  torr,  and  is 

referred  to  as  white  noise. 

3.  The  process  is  also  refeired  to  as  a  memoryless  process. 
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5  12  3  4 

Fig.  5.1.  Theoretical  autocovariance  of  a  pure  random  process. 


Fig.  5.2.  Theoretical  spectrum  ot  a  pure  random  process. 

52.1 . 1  Example  of  a  Pure  Random  Process.  Figure  5.3  shows  an  example  of  a 
pure  random  Gaussian  process  generated  for  r  =  1  —  600,  with  average  0  and  variance 
1.0.  At  the  bottom,  a  part  (r  •-  100  —  250)  of  the  record  is  shown  expanded  on  the  time 
axis.  Its  readings  are  listed  in  Appendix  A1  as  Table  Al.l;  pp.  251,  252,  and  253.  for 
reference.  This  process  was  generated  by  the  method  of  “Random  number  generation  and 
testing,”  generally  popular  as  “multiplication  type  residual  method.” 

It  was  found  that  by  this  method  it  is  hard  to  get  a  really  while  process  with  precise¬ 
ly  designed  variance  of  o}  =  1 .0  from  this  short  record  {N  =  600),  and  the  sample 
variance  appeared  as  af  =  1.046490.  Figure  5.4a  shows  the  theoretical  autocorreiation 
coefficient  p(0)  = /f(0)//f(0)  =  1,  p(r)  = /?(r)//?(0)  =  0,  r  ^  0  by  Eq.  5.5.  Figure  5.4b 

shows  the  estimated  p(r)  =  R{r)/R{0)  from  the  generated  process.  From  the  model  fitting 
technique  and  order  determination  that  will  be  mentioned  in  Section  5.5,  the  order  ap¬ 
peared  to  be  0,  and  AR(0)  appeared  to  be  the  most  appropriate  model  to  fit  this  process. 
The  estimated  autocorrelation  of  this  model  fitted  by  Eq.  5.4  is  just  the  same  with  the 

theoretical  p(r}  shown  as  Fig.  5.4a  because  the  difference  is  only  in values,  that  its 
drawing  was  omitted.  Figure  5.5a  shows  the  theoretical  spectrum  function  of  this  process 
designed  by  Eq.  5.6,  that  is,  the  white  noise.  Figure  5.5b  shows  the  estimated  spectrum  of 
the  model  fitted  by  Eq.  5.6.  This  is  also  the  white  noise  and  looks  very  similar  to  the 
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Fig.  5.3.  Simulated  pure  random  process  AR(0) 

X,  =  €„  €,:N[0.1]. 

theoretical  spectrum  except  that  the  variance  was  a  little  larger,  ai=  1.046490,  as 
mentioned  above. 

In  Fig.  5.5c,  for  comparison,  the  estimated  spectrum,  calculated  by  the  nonparamet- 
ric  or  correlation  method,  and  the  Fourier  transform  of  the  calculated  ^(r)  in  Fig.  5.4b  are 
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0.00  7.50  15.00  22.50  30.00  37.50  45.00  52.50  60.00 


Fig.  5.4a.  Theoretical  AR(0). 


Rg.  5.4b.  Estimated. 

Rg.  5.4.  Autocorrelation  coefficient  for  pure  random  process  AR(0) 

=  e,;N[0.1I. 

shown.  The  maximum  lag  number  Af  =  60  and  the  so-called  Hanning  Type  spectral  win 
dow  was  used  to  calculate  this  spectrum.  It  looks  very  different  from  the  white  spectrum 
and  has  many  sizable  peaks  and  valleys.  In  the  figures,  the  9C%  level  of  confidence  inter¬ 
val  of  this  estimate,  based  on  the  -distribution  of  the  equivalent  degree  of  freedom 
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(here  approximately  equal  to  27).  as  is  explained  in  Section  2.5.5  is  shown  as  vertical 
lines  for  reference. 
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Fig.  5.5a.  Theoretical  AR(0). 
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Fig.  5.5b. 
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Estimated  by  model  fitting  as  optimum  AR(0). 


Rg.  5.5.  Spectrum  for  pure  random  process  AR(0) 

X,  =  e,.  €,:N[0.1]. 

This  difference  tells  us  that  the  process  might  not  actually  be  white  in  this  short 
period  of  600  and  also  that  it  is  hard  to  get  the  'real'  character  of  the  spectrum  by  the 
nonparametric  method  from  this  short  record.  It  is  interesting  to  note,  however,  that  by 
the  model  fitting  method,  even  from  this  short  record,  the  fluctuations  disappeared  and 
the  spectrum  appeared  to  be  white. 
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Rg.  5.5c.  Estimated  by  nonparametric  method. 
Pig.  5.5.  (Continued) 


522  Autoregressive  Process  of  the  First  Order,  AR(1 ). 

When  X,  follows  the  relation 

X,  =  oXm  +  e, ,  (5.7) 


where  a  is  a  constant  and  le,|  is  a  stationary  pure  random  process. 


then 


ai  when  r  =  0 
0  otherwise. 


(Fig.  5.7)  (5.8) 


{.JQ )  is  called  an  autoregressive  process  of  the  first  order  AR(1).  Equation  5.7  means  that 

1 .  Xt  has  a  linear  regression  on  X(-i  as  in  Fig.  5.6 

2.  e,  plays  the  role  of  error  in  the  above  relations  of  regression 

3.  X,  is  dependent  on  one  step  back  value  of  the  same  process  X,^i. 
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X, 

t 


Fig.  5.6.  X,  vs.  X^. 

Besides,  when  |X, }  is  Gaussian,  this  process  is  called  a  “Linear  Markov  Process’"  as 
will  be  explained  in  Part  ni. 


fi-i 


Fig.  5.7.  e,  vs. 

522.1  Green’s  Function  cfAR(l). 

Xf  =  €j+ 

=  €(+  a(€  ,_i  +  aXt-2) 


=  €,+  a  €,-i  +  -a^  €t-2+  ■  ■  ■  +  a‘ 

r-l 

=  a''  €,.^+a'  Xo. 

r  =  0 


^|ei  +  aXo| 


(5.9) 


If  initially  Xo  =  0,  or  X,.  =  0  when  r  s  0, 
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then 


(5.10) 


or,  considering  the  stationarity  of  X,  and  assuming  that  X_,  exist, 

t 

Xt=  X  a +  a .  If  isO,  when/Vtends  to  oc 


M  =  -  (^-1) 


x,=  X  Z 

u  =-00  )  =  0 


which  is  the  same  form  as  Eq.  5.10.  Setting 


Gj  =  aj 


(5.11) 


(5.12) 


Xr-^Gj€t.j=  X  (5.13) 

j  =  0  )  =  -00 

Gj  is  called  Green’s  function  for  AR(1 ). 

Green’s  function  Gj  shows  the  weight  given  in  the  present  response  of  A)  to  the  dis¬ 
turbance  6^.^  ,  which  entered  the  system  j  time  units  back.  It  also  indicates  how  well  the 
system  remembers  the  disturbance  6f_y  or  bow  slowly  or  quickly  the  dynamic  response  of 
the  system  to  any  particular  e,_y  decays.  Equation  5.13  is  called  Wold’s  decomposition 
and  gives  the  decomposition  of  X,  into  an  infinite  number  of  orthogonal  variables  Gj  c^.y . 
Equation  5.13  also  implies  that  the  AR(1)  process  can  be  inverted  into  an  infinite  order 
moving  average  process  MA  ( » )  as 

Xf  =  Go  €t+Gi  €f_i  +  G2  6/_2+'  •  •  +  Gy  •  ■  •  (5.13’) 

522.2  Solution  of  the  Difference  Equation.  Equation  5.7  is  a  difference  equation 
and  can  be  written 

X,-aX^.i  =  €,.  (5.7’) 

The  general  solution  of  the  difference  equation  is  the  sum  of  the  solution  of  its 
homogeneous  equation  plus  a  particular  solution  of  Eq.  5.7’.  The  homogeneous 
equation  of  Eq.  5.7’  is 

Xt-aX^i  =  0,  (5.14) 


its  solution  being 

Xi  =  Afi\ 

where  u  is  the  root  of  its  characteristic  equation,  equated  to  zero. 


(5.15) 


namely 


f{Z)  =  Z-a  =  0, 
fi  =  a. 


(5.16) 

(5.17) 


is  an  arbitrary  constant  and  is  determined  by  the  initial  conditions.  From  Eqs. 

5.15  and  5.17, 

X,  =  Aa'.  (5.18) 

One  particular  solution  of  Eq.  5.7  is  obtained  as  follows  using  the  backward  shifting 
operator  B.  Equation  5.7  is 

{\-aB)  X,  =  €,,  (5.19) 

00  00 
X,  =  (1  -oBT^e,  =  X  =  Z 

;=0  j=0 

This  equation  is  the  same  as  Eq.  5.10  or  5.13  expressed  by  Green’s  function.  Therefore, 
the  general  solution  of  Eq.  5.7  is 

GO 

X,~Aa‘+'2^^  (5.21) 

>  =  0 

The  first  term  that  is  the  solution  of  the  homogenous  equation  is  the  free  oscillanon  of 
this  system.  When  lal  <  1,  this  free  oscillation  decays,  and  only  the  second  term  remains 
as  a  stationary  oscillation  as  in  Eq.  5.20.  Equation  5.19  is,  more  generally, 

aiB)X,  =  €,.  (5.22) 

X,  =  a-HB)€,,  (5.23) 

where 

a(Z)  =  l-aZ.  (5.24) 

In  order  to  have  the  free  oscillation  damp  out,  lal  <  1  is  necessary  and,  with  the  character¬ 
istic  Eq.  5.16  equated  to  zero,^20  =  0  must  have  its  root  inside  the  unit  circle.  Then  from 
the  equation 

a(Z)  =  0,  (5.25) 

or  1  -  aZ  =  0,  its  root  is  the  reverse  of  the  root  of  the  equation  Z  -  a  =  0,  or  fiZ)  =  0.  We 
find  a(Z)  =  0  must  have  its  root  outside  the  unit  circle. 

5223  Inverse  Function  of  AR( I).  Green’s  function  can  be  considered  as  an  indi¬ 
cation  of  how  X  can  be  expressed  by  the  MA  ( » )  process,  because  Xi  is  expressed  as  the 

summation  of  infmitee,  at  preceding  time  points. 
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e,= 

>  =  o 

=  - IjBJ  +  -  ■  ■)X, 

=  -IoX,-h  X,.i~l2  X^2 - Ij  X,.j - ,  (5.26) 

functions •  •  Ij  are  called  inverse  functions,  and  usually  -/o  =  1 . 

Therefore 

€,  =  X,-hXt.i -hX^2 - IjX,.j-  •  • .  (5.27) 

This  inverse  function  shows  how  A)  can  be  expressed  by  AR  ( » ) .  Therefore  the 
inverse  function  of  a  pure  autoregressive  process  AR(n)  actually  has  no  meaning.  For 
example,  for  AR(1)  as  in  Eq.  5.7; 

/o  =  - 1 ,  /i  =  +  a,  lj  =  0  fory  S  2.  (5.28) 

522.4  StationalityofAR(I).  From  Eq.  5.10 


/- 1 

X,  =  lim  X  +  •  •  • 


[-1 


ei 


00 


j  =  0 


Then  if 


£[ft]  =f4e  for  all  t. 


and  as  €[  represents  white  noise. 


El€t€u] 


a},  t  -  u 

0,  t^u 


we  can  get 

I 


(5.29) 


(5.30) 
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=  lim  |Ue  1 1  +  +  •  •  •  +  a'  ^ 

•  00  ^ 


=  lim  i 
/-•  00 


1  -a' 


1  -a 


fii  a 


(5.31) 


fift  a  =  1 


cov.  [A:,  =  lim 

f-fcoe 


.)(i 

;  =  0 


isO 


=  lim  <7f  ■ 


Therefore 


'>  r 

OTa 


R(r)  =  lim  ^ 

/-►oe 


l-a^' 
1-^  *’ 


L  oit. 


a 


0=1 


(5.32) 


lf/ie  =  0. 


var.  pej  =  /?(0)  =  lim 

l-*o> 


2  1  -  O^ 

7€- - f. 

1  ~a^ 


pit. 


a^\ 


0=1 


(5.33) 


From  these  results: 

[  a.  ]  When fi  0 

E[Xt\  is,  from  Eq.  5.31,  a  function  of  time  and  therefore  (Ail  is  not  stationary',  even 
to  order  1. 

(1 .)  When  lol  <  1 ,  and  when  r  tends  to  a  large  value 


E[XJ  =  =  const,  by  r. 

1  -o 

Therefore  (X,)  is  asymptotically  sutionaiy  to  order  1. 


(5.34) 


(2.)  When  o  =  1 


E[X,^=Htt  (5.35) 

£[x]  increases  by  time  r.  Therefore,  (X,)  is  not  stationary,  even  up  to  order  1.  This  is  the 
case  of  Random  Walk. 

[ b.  ]  When (1^=0 

£[XJ  =  0. 

cov.  |X,  •  Xt+r]  =  R(r)  =  aia'' 
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(1.)  When  lal  <  1,  i.e.,  wheny(Z)  =Z-a  =  0  has  its  root  inside  the  unit  circle  or 
whena(Z)  =  1  - oZ  =  0  has  its  root  outside  the  unit  circle,  from  Eq.  5.32,  even  £i  A', )  =  0 
holds,  the  cov.  [A„  J  is  a  function  of  time.  Accordingly,  {A, )  is  not  staiionaiy’  even 
up  to  order  2.  However,  when  r  , 


cov.  [X,  X,^,]  =/?(r)  =  -— 

1  -a~ 

const,  by  r. 

(5.36) 

var.  [X,]  =  £[X,  X,]  =  R(0)  = 

l-a‘ 

const,  by  t. 

(5.37) 

Therefore,  {A,}  is  asymptotically  stationary  up  to  order  2.  The  form  of  R(r)  will  be 
different  by  the  sign  of  a.  although  it  gradually  decays  as  r  increases  as  in  Fig.  5.8. 


R(r) 


f 

y 

-r-n 

N 

ri-r . 

R(r) 


-2-10  1  2  3 


(a)  0  <  a  <  1 


Fig.  5.8.  H(r)ofAR(1). 


(2.)  When  kri  >  1,  i.e.,  when  J(Z)  =  Z-a  has  its  root  outside  the  unit  circle,  or  when 
a(Z)  =  1  -  <zZ  has  its  root  inside  the  unit  circle, 

r2 


cov.  [A,  •  A,+r]  =  Rir)  -  1) 


a^~\ 


var.  [XJ  =  {a^- 1). 


(5.38) 

(5.39) 


In  this  case,  £[A]  =  0,  but  not  only  does R(r)  not  converge  to  a  small  value  as  r  increases, 
but  both  R(r)  and/?(0)  change  by  the  time  r  and  are  not  stationary.  As  time  passes,  these 
values  continue  to  increase.  Accordingly,  this  process  is  not  stationary  up  to  order  2,  even 
asymptotically. 

52.25  Autocovariance  and  Spectrum  ofAR(l).  The  general  solution  ofi?(r)  can 
also  be  obtained  more  simply  as  the  solution  of  a  first  order  homogenous  difference  equa¬ 
tion,  because  in  this  case,  from  Eq.  5.7, 

X,  =  aXM  +  e,,  (5.7) 

assuming  ir  c  =  0  and  £[Ai]  =  0.  Multiplying  both  sides  of  Eq.  5.7  by  A,_  ^  and  taking  the 
expected  values  gives 


R(r)  =  a  R{r-  1) 
RiO)  =  a\R(Q)  +  ai 


when  r 


(5.40) 


because 


when  r  =  0, 


E[Xr€j]  = 


0 

'  E[€f]  =  a} 

h 


‘  ^  j 
i=j\ 


(5.41) 


(5.42) 


Equation  5.40  is  the  same  type  of  homogeneous  difference  equation  as  that  of  the  original 


process  (X,)  shown  in  Eq.  5.14. 

R(r)-aR(r- 1)  =  0.  (5.43) 

Using  the  backward  shift  operator  B  gives 

(\-aB)R(r)  =  0.  (5.44) 

Thus  R(r)  =  (1  -oBT^Rir)  =  0  (5.45) 

and  R{r)~A^\  (5.46) 

Here/z  is  the  root  of  /fZ)  =  Z-a  =  0  when  fi  =  a 


and  thus  R{r)  =  Aa'^.  (5.47) 

In  order  to  be  asymptotically  stationary,  R(f)  -*•  0  as  (r)  -*  c»  when  lal  <  1  and^^Z)  =  0 
must  have  its  roots  inside  the  unit  circle. 

From  Eq.  5.41 

i?(0)  =  (7|=i4  =  — (5.48) 
Therefore,  the  general  solution  is,  from  Eq.  5.43, 

R(r)  =  — (5.49) 
1  -tr 


This  equation  is  the  same  as  Eq.  5.36. 

522.6  sift))  ofAR(l).  Here  we  assume =  0,  la)  <  1. 

Then  the  spectrum  s{(o)  is  obtained  as  the  Fourier  transform  of  R{r) 

2^  r=-« 


_1_ 

2ji 


]/?(0)  +  2  X/fCr) 


cos  r(o  1 . 


no 


Substituting  the  values  of  ^(0)  and  R{r)  in  Eqs.  5.48  and  5.49  gives 

1 


=  — Ox  ^  1  +  2  y  ^t'^cos 
2^  >  .=1 


0 

a: 


r(o 


2jr 


27t 


7jt 


00 

'  ] 

1  +  2  y  Sie 

^re^irw 

r=  1 

J 

1  +  2  iae‘^  +  (ae‘^)^  +  (ae‘^)^  +  ■ 


1  +  2  9le 


ae 


■  uo 


231 


o'^ 


1  +  2  9le  ^ 


a  COSO)  +  la  smw 


25r 


1  +  2 


(1  -fl  cos  (o)-i  asm  (o 
acosa)(l  -acosa))-a^sin^tu 


1  -t7 


(1 -acosG))^  +  a^sina)  [  2jt  I -lacosw  +  ' 


here  SRe  (  • }  indicates  to  take  the  real  pan  of  a  function  (• } . 


Inserting  Eq.  5.37,  that  is  ajf  = - - ,  gives 

1  —  <7* 


S{(0)  = 


oi 


1 


Tjt  (1  -2<zcosa/  +  <7^) 

o} 


oi 


Tjt 


1  -ae~^ 


2jt 


a{e-^\ 


(5.50) 


Therefore, 


5(0)  = 


1  -(?■  Or  1  +a 


23t  (1  -a)^  2ji  1-a 

oi  1 


2;r  (l-a)^’ 


(5.51) 
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5(;r)  =  si-  Jt)  = 


1  -a  o; 

\  +  a  2ji 


o}  1 

2ji  {\-»ra)~ 


(5.52) 


Its  shape  will  then  be  as  shown  in  Fig.  5.9. 


S((*  )/S(D| 


Fig.  5.9.  s(a))of  AR(1). 


522.7  Estimation  of  a  and  a} .  When  we  are  given  a  set  of  data,  Xi  •  .  with 

numbers  N,  and  the  AR(1)  model  is  to  be  fitted  (the  determination  of  order  will  be  dis¬ 
cussed  later  in  Section  5.5),  we  can  easily  estimate  the  values  of  a  andof  by  the 
minimum  least  squares  method  as 

N 

S  Xt  x,.i 

a  =  ^ -  (5.53) 

txU 

1  =  2 


and  the  minimized  residual  sum  is 

1  ^ 

o}  =  X  ^  X,.,if.  (5.54) 

The  same  results  can  also  be  derived  from  Eq.  5.40  and  Eq.  5.41  as 

.  ^  ^(1) 

^  RiO)  <5.55) 

and  6i=il-a^M0).  (5-56) 

Equations  5.53  and  5.55  have  the  same  content  as  if  the  sample  correlation  were  replaced 
with  f?(0)  and  /?(!).  Obtaining  n  by  the  linear  minimum  least  squares  method  from  Eq. 
5.53  is  actually  done  by  Eq.  5.55.  Then,  the  variance  a  ^  ofe,  is  estimated  by  Eq.  5.56. 

522.8  Example  ofAR(l).  Figure  5.10  is  an  example  of  the  AR(1)  process  simu¬ 
lated  by  an  AR(1 )  model  AT,  -  0.5  AT^.]  =  e,,  where  a  =  -  0.5  in  Eq.  5.7.  In  order  to  look 
at  the  pattern  of  variations,  at  the  bottom  of  this  figure,  a  part  (r  =  100  to  250)  of  this  pro¬ 
cess  is  shown  expanded  on  the  time  axis.  Its  readings  are  listed  in  Appendix  A1  as  Table 
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5.00  ^ 


-5.00-^ - ^ ^ - - - - - - 

0.00  75  00  150  00  225  00  300.00  375.00  450  00  525  00  600  00 


5.00-) 


A1.2;  pp.  251, 252,  and  253.  Herec,  is  a  pure  random  Gaussian  process  N[0,  IJ.  and  the 
same  process,  which  was  generated  as  a  pure  random  process  AR(0)  in  Fig.  5.3,  was 
used.  Figure  5.11a  shows  the  theoretical  autocorrelation  coefficient  p(r)  =  R{r)/R(0), 

from  Eqs.  5.36  and  5.37  using  the  designed  value  of<7|=  1  and  a  =  -0.5.  Figure  5.11b  is 

A  A  A 

the  estimated  autocorrelation  function  p(r)  =  R{r)/R(0)  from  the  simulated  process  in  Fig. 
5.10.  An  AR  model  was  fitted  to  this  process,  and  the  order  actually  obtained  was  1  by 
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Fig.  5.11b.  Estimated. 

Rg.  5.11.  Autocorrelation  coefficient  for  AR(1 )  process 
X,-0.5X„=e,.  €,:N10,1]. 


the  method  of  order  determination  that  will  be  mentioned  later  in  Section  5.5.  Now  with 
the  order  as  1 ,  a  anda^  were  calculated  from  Eqs.  5.55  and  5.56,  as  a  =  -  0.50933  and 
CTe  =  1 .04646,  which  are  close  to  the  a  =  -  0.5  anda^  =  1.0  used  to  generate  the  process. 
We  also  estimated  the  correlation  coefficient  p(r)  =  /?(r)//?(0)  from  Eqs.  5.36  and  5.37, 
using  these  a  andde  .  These  values  are  so  close  to  the  theoretical  /?(r),  or  just  the  same  in 
p(r)  =  R{r)/R(0)  in  Fig.  5.11a  that  their  drawing  was  omitted.  Figure  5.12a  is  the  theoret¬ 
ical  spectrum  from  Eq.  5.50,  using  the  design  values  of  a  =  -0.5  and  af  =  1 . 


0) 


Rg.  5.12a.  Theoretical  AR(1). 

Rg.5.12.  Spectrum  for  AR(1)  process  X,- 0.5  Xm  =  e,,  e,:N[0,1]. 

The  estimated  spectrum  s((w),  with  a  =  -0.50933  andof  =  1.04646  obtained  by 
model  fitting  and  also  by  Eq.  5.50,  is  given  as  b  in  Fig.  5.12.  For  comparison,  as  c  in  Fig. 
5.12,  the  spectrum  s{(o)  was  also  estimated  by  the  nonparametric  method  as  the  Fourier 

transform  of  R{r)  in  Fig.  5.11b,  using  the  Hanning  window  and  maximum  lag  M~60.  It 
is  interesting  that  the  spectrum  estimated  by  model  fitting  is  very  similar  to  the  theoretical 
spectrum.  Fig.  5.12a,  of  this  generated  process.  On  the  other  hand,  the  spectrum  esti¬ 
mated  by  the  nonparametric  method  Fig.  5.12c  is  more  wavy  than  the  theoretical  one, 
although  the  fundamental  shape  is  similar.  The  wavy  fluctuations  were  found  to  come 

mostly  from  the  wavy  fluctuation  of  the  input  white  noise  {6t }  that  is  shown  in  Fig.  5.5c, 
as  the  incomplete  whiteness  of  the  pure  random  process  {e/ }  generated.  In  the  figures. 


Fig.  5.12b.  Estimated  by  AR  model  fitting  as  optimum  AR(1). 


Fig.  5.12c.  Estimated  by  nonparametric  method. 
Rg.  5.12.  (Continued) 


the  90%  level  of  confidence  interval  of  this  estimate,  based  on  the  -distribution  of  the 

equivalent  degree  of  freedom  (here  approximately  equal  to  27)  as  is  explained  in  Section 
2.5.5  is  shown  as  vertical  lines  for  reference. 

S2J  Autoregressive  Process  of  the  Second  Order.  AR(2) 

As  was  shown  in  the  last  section  for  the  AR(1)  process,  the  residual  part  e,  of  X,, 
obtained  by  subtracting  the  linearly  dependent  part  from 

e,  =  X,-dK,.i  (5.7’) 

was  a  purely  random  process  uncorrelated  withe,_i .  f /-2  •  •  •  as  in  Fig.  5.13a  and  xmcor- 
related  with  A)_2,  Ai_3  •  •  •  as  in  Fig.  5.13b. 


Rg.  5.13a.  c,  vs.  for  AR(1). 


et 

I  •.  .*  •,  • 

01 - 

X,-2  — 

Fig.  5.13b.  e,  vs.  X«forAR(1). 
Rg.  5.1 3.  Characters  of  €/  for  AR(1 ). 


0 


r  =  0 
r 


E[erX,_2]  s  0. 


If  this  relation  does  not  exist  and  the  residual  ,  that  is, 

e/  =  X,-a'XM, 

was  linearly  correlated  with  X^2  as  in  Fig.  5.14,  then 


(5.57) 
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(5.58) 


=  a2X,^2+i,- 

The  residual  part  e,  =€,'  -ai  X^2  is  now  a  purely  random  process.  Then  inserting 
Eq.  5.57  into  Eq.  5.58  gives  Xt  -  -  ai  X^2  -  f  /  • 


e; 

t 

\-2 

Fig.  5.14.  € ;  vs.  X,_j  for  AR{2). 

Here  changing  -a'  =  ai,  -  ai-*  <22, 

Xi  +  a  iX/_i  +  02X1^2  =  f  /  •  (5.59) 

The  process,  which  satisfies  Eq.  5.59,  is  called  the  autoregressive  process  of  the  sec¬ 
ond  order.  Here,  ^2]  and  <32^6  constant,  {e,}  is  a  stationary  pure  random  process,  and  we 
assume  £[e,]  =  0. 

52  JJ  Solution  of  AR(2),  Green’s  Function  for  AR(2).  Starting  from  Eq.  5.59 

Xt  + a  iXf^i  +  02X1-2  =  f  f . 


we  can  solve  X)  as  the  general  solution  of  a  second  order  difference  equation,  i.e.,  a 
general  solution  equals  a  complementary  function  plus  a  particular  solution.  A  comple¬ 
mentary  function  is  the  solution  of  the  homogeneous  equaticm 


Xt  +  OiXf-i  +  02X1-2  —  0. 

(5.60) 

The  solution  of  this  homogeneous  equation  is  in  the  shape  of 

M  +^2 

(5.61) 

fi\,fi2i  being  the  roots  of  the  characterisitc  equation /(Z)  =  0.  The  particular  solution  can 

be  obtained,  using  the  backward  shifting  operator  B,  as 

(1  +  0\B  02B^'Xt  —  • 

(5.62) 

Setting 

a(Z)  =  1+  ojZ  +  02Z^, 

(5.63) 

a(B)Xt  =  €,. 

(5.64) 

If  (he  quadrant 

f(Z)  =  Z^  +  oiZ  +  O2Z^  =  0 

(5.65) 

has  two  roots 
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1  -  =  6;  , 


(5.66) 


^1^2  =  ai. 


then 


(l-Al5)(l-/^2OT  =  e,, 


Xr  =  — ^  = 


1 


cl{B)  (1  +  +  ^2-®^)  (1  ~  fi\B){\  —  fX2B) 

1  f  _ fii 

fll-^l2\{l-^llB)  (1-//2®) 


=  -^j  I 

{f: 


r  =  0 


r 

-  •< 


A)  is  a  particular  solution.  Therefore,  if  the  expression  of  Oreen’s  function  Gy  is 
used, 

00 

j=0 


Thus 


-f^2  M'l  ^2  -f^\ 


=  51  f^i+82fii 


where  = 


52  = 


/^l-/*2 

^2 

A2-/^1 


(5.67) 


(5.66’) 


(5.68) 


(5.69) 


(5.70) 
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(5.71) 


This  solution  X,  can  also  be  written  as 
» 

X,  =  ^  =  Gof/+Gie,_  1  +  G2e,_2+ ■  •  • 

j  =  0 

Equation  5.71  shows  the  AR(2)  process  is  expressed  by  MA(  oo ),  as  was  the  case  for 
AR(1). 

52  J2  Inverse  Function  for  AR(2f.  To  say  the  inverse  function  for  the  autoregres¬ 
sive  process  has  little  meaning  was  pointed  out  for  AR(1).  Formally,  however,  for  Ak(2), 
ffcnn 

Xt  +  aiX,^i  ■¥  02X^2  =  f  / . 

00 

(1  aiB  +  a2B^)X,  =  ^  (-/,)X,_  ■ 
y=o 

=  (-/o-/iB-/252.  •  •  IjB^)X,  .  a  -  00) 

Therefore 

li=-ai 

h  =  -a2 

Ij  =0  for  ;■  S  3.  (5 j2) 

5233  Stationality  ofAR(2).  Returning  to  the  general  solution  of  X,  we  must  con¬ 
sider  the  behavior  of  the  complementary  function  or  the  solution  of  the  homogeneous 
Eq.  5.60  that  is  in  the  shape  of 

Ai  h{-^A2  fl2- 

This  complementary  function  represents  the  free  oscillation  of  a  forced  oscillation  system 
which  should  die  out  over  time  r  and  become  asymptotically  stationary.  For  this  condi¬ 
tion  it  is  necessary  that 

l^tll  <  I,  (^2l  <  I- 

This  means  that  the  quadrant,  as  expressed  by  Eq.  5.65  equated  to  zero, 

fi7)  =  Z^  +  aiZ+a2  =  0  (5.73) 

should  have  roots  less  than  1  in  their  absolute  values.  ThCTefore,  fiZ)  =  0  must  have 
roots  inside  the  unit  circle.  These  roots  of  fiZ)  are  the  reciprocals  of  the  roots  of 

a(Z)  =  1  aiZ  -«•  a2Z^.  (5.74) 

Accordingly,  a(Z)  must  have  roots  outside  the  unit  circle  in  order  for  AR(2)  to  be  asymp¬ 
totically  stationary. 
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fiZ)  =  0,  i.e., 


Z^  +  <2lZ  +  <l2  —  0 


has  two  roots 


(5.75) 


(5.76) 


-fli  ± 

fiufii  - - 

aod  these  two  roots /ti,  fi^  satisfy  Eq.  5.67.  Therefore,  from  the  condiiion 

(uil  <  1,  (wjl  <  1, 

1  >  02  >  -  1 

and,  as  fii<fi2<  1, 

Ai(l  -Ml)  <  1  -Ml- 
Thus  Mi-^Mi-Mi  Mi<) 

and  _  <  1 

0l+02>“l,  (02>“fll-l) 

and  as  itj  >  - 1,  <1,  - 1  <  ^2  <  1 

-0^-*‘Ml)Mi<(\-^Mi)- 

Thus  -  (^1  +  Ml)-MiMl  <  1 

and  oi-a2<l,  (a2>ai-l). 

Therefore,  as  shown  in  Fig.  5.15,  on  a  0  -  oi,  02 plane,  the  region  inside  of  A  ABC  is  the 
stable  zone  for  ai  and  02-  This  stable  zone  is  divided  into  two  subzones. 

1.  When  Eq.  5.73  has  two  real  roots  or  coincident  equal  roots, 

1  ^  ‘  -  1 


(5.77) 


(5.78) 


af  S  4o2  or  02  S 

This  is  subzone  [  I  ]  in  Fig.  5.16. 

2.  When  Eq.  5.73  has  unequal  complex  roots, 

1a  ^2 

ai<4o2,  or  a2>—af. 
This  is  subzone  [  n  ]  in  Fig.  5.16. 


(5.79) 


(5.80) 
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Fig.  5.16.  Stable  Subzones  1 1  ],  I II  ]:  (^,  (^. 

523.4  Autocovariance  ofAR(2).  From  Eq.  5.59 

Xt  +  aiXi^i  +  a^i_2  =  e , 

assuming  £[X,]  =  0, 


taerfj-w]  =  ^(r)a/=  ^  ^  ^ 

^  ai  r  =  0 

k 


d(r)  is  Kronecker’s  delta  function. 


From  Eq.  5.59 


E[Xr^€r] 


* 

0 

'  £[€?]=< 
k 


r  ?£  0 
r=  0 


ElXi  +  ai£l[X/_|  X/_^]  +  02  £[X/_7  *  =  E[€t  •  X«_^]. 


Therefore,  when 


r  =  0 

:  Ri0)  +  axR(l)  +  O2R(2)=a^ 

(5.82) 

r=  1 

:  Ril)  +  OiRi0)  +  O2Ril)  =  0 

(5.83) 

r  =  2 

;  /?(2)  +  ai/2(l)  +  a2/?(0)  =  0 

(5.84) 

r  S2 

generally  R{r)  +  0\R{r  -  1 )  +  02R{r  -  2)  =  0. 

(5.85) 

From  Eq.  5.83 


and  therefore 


/?( 1 )( 1  +  ^2)  =  -  ai^(O)  = -a  lal 
RkO)  =  oI 


R(l)  =  1  — ^  1  al 
1+02 


Inserting  this  value  into  Eq.  5.84  gives 


/?(2)  = 


1+^2 

Substituting  R(2)  into  Eq.  5.82,  gives 


- 02  iai. 


ai  = 


{l+02)oi 


(1  -02X1  -ai  +  <12X1  +ai  +  02) 


This  equation  connectsci  with  Of.  From  Eq.  5.86 

-oi 


R(l)  = 


(1 —O2XI —Oi +02X1  +  Ol  +  O2) 


From  Eq.  5.87 


(5.86) 


(5.87) 


(5.88) 


(5.89) 


I+O2 


+  01-02-02 


(1  -  02X1  -01+  O2KI  -ai  +  02) 


Cfi 


(5.90) 


Equation  5.85  is  a  homogeneous  equation  with  the  same  coefficient  as  the  homogeneous 
Eq.  5.60.  Its  solution  is  in  the  shape  of 
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R{r)  =  Bifi\  +  B-2M2^ 


(5.91) 


H\,fX2  being  the  roots  of  Eq.  5.73,  f{Z)  =  Z'  +  niZ  +  02  =  0.  8^,82  are  constants  and  are 
determined  by  the  boundary  conditions,  as 


RiO)  =  al 


Thus 


Bi+B2  =  a;  and  Bifi\  + 82/^2  = 


-a\ 

1  +02 


ol 


Therefore 


Cai-;«2)(1 +Al^2) 

52  =  ^2(1 -Aif)  ^2 

(At2-/fi)(l+AiiA^2) 


Then  /?(r)  = 


-M 


d'A^iV' 


(A4l  -At2)(l  (Ml  -A2)(1  +M1M2) 


Mi 


ol 


(5.92) 

(5.93) 


(5.94) 


If  we  need  the  expression Ce  from  Eq.  5.88 

1+02 


(1  -02X1  -  Ol  +  02X1  +01  +  02) 
1  +fl\fl2 


0} 


then 


R(r)  = 


(1  -H^H2)0-  -MXl  -MV 


- -  u{ - ^ - ui 

iMi-M2){^-Mh^-MiM2)  (Mi-M2)(\-Mb(y-MiM2) 


(5.95) 


a}.  (5.96) 


This  equation  is  in  the  form  of  R{r)  =  B\  n\  +  82 

There  are  a  variety  of  cases  in  which,  depending  on  the  sign  oifii,H2,  the  autoco¬ 
variance  function  appears  different  in  its  tendency. 

1.  When 02  ^  oiy4  (subzone  [I]  in  Fig.  5.16),  H\,fi2  are  real,  and 

a.  if  Ol  <  0,  02  >  0  (region  ^  in  Fig.  5.16),  H\>0,  02  >  0 

andi?(r)  stays  positive  as  in  Fig.  5.17(A), 

b.  if  Ol  <  0,  02  <  0  (region  in  Fig.  5.16),  then  ^1,02  have  opposite 

signs. 
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c.  if  ai  >  0,  ai  >  0  (region  in  Fig.  5.16),  then;/)  <0.  M2<0  and  in 

this  case Rir)  alternate  signs  by  r  as  in  Fig.  5.17(B),  and 

d.  if  fl]  >  0,  02  <0  (region  in  Fig.  5.16),  then  have  opposite 

signs. 

Therefore,  for  a  and  c,  the  autocovariance  function  shows  the  patterns  (A)  and  (B) 
in  Fig.  5.17,  respectively.  For  b  and  d,  the  shape  will  be  (A)  or  (B),  depending  on  the 
size  of  fi\,fi2 . 


I  R(0 


R(r) 


f 


(A):  a 


R(t^  of  AR(2);aj  >  ^ 
4 


\\  >  *  •■jsrs- 


'J'  ^ 

_ 


(B):  c 


(A)  :  a,  (a,  <  O.aj  >  0)  (B)  :  c,  (a,  >  O.aj  >  0) 

ng.  5.17.  R{t)  of  AR(2);  a*  s  a? /4. 

2.  Whenaa  >  Oy/A,  (subzone  [11]  in  Fig.  5.16),  fiufi2  are  complex  and  conjugate  to 
each  other,  and  from  Eqs.  5.67  and  5.75 

fulfil- )Ui  =  y^e'®'  fjL2  = 


and-ai  +;/2.  which  satisfies 

-ai  =  ij^cosa. 

Thus  cos  a  =  -  ai/2  7^ 

and  fii-fi2  =  2j^isma. 

Using  the  relations  of  Eqs.  5.97  to  5.99  in  Eq.  5.94  gives 

|sin(r+  1)  <7-a2sin(r-  Ijaj 


(5.97) 

(5.98) 

(5.98’) 

(5.99) 


Rir)  =  ai^- 


Here  again  setting 


tan0  = 


(1  +a2)sina 
1+^2 


ol 


1  -02 


tana. 


(5.100) 


gives 
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R{r)  =  ^2^^ 


sin(ra  +  0) 
sin  6 


>ol 


(5.101) 


Equation  5.101  shows  that  R{r)  has  the  form  of  a  damping  periodic  function  with  period 
In/o  and  damping  ao  . 

From  Eq.  5.98’ 

a  =  cos"*^-  ai/2  (5.98”) 

so  02  determines  the  damping  and  a^/Toz  determines  the  period  (see  Fig.  5.18). 


As  an  extreme  case  of  stationality,  when <32  -  1,  from  Eq.  5.1000  =  n/l. 
Therefore  from  Eq.  5.101 


R{r)-*  cos  ra  al  (5.102) 

This  is  an  undamped  cosine  curve  and  shows  that^(r)  does  not  decay  with  increasing  r 
but  continues  to  oscillate  in  this  case. 


5JJJ  Estimation  ofoi,  02-  From  Eqs.  5.83  and  5.84 

faii?(0) +  <22^(1)  =  -/?(!) 
ai/?(l)  +  a2i?(0)  =  -i?(2) 


A 

^(1)  ^(1) 
Ri2)  R(0) 

/?(0)  R{\) 

R(i)  Ra) 

at  = 

’  G2  ~ 

R{0)  R{1) 

R{0)  R{1) 

m)  m) 

RKD  R(0) 

(5.103) 


or 
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(5.103’) 


.  R{1)  R{2)~R(0)  R{1) 

di  = - 5 ^ - 


.  /?-(!) -/?(0)/?(2) 

rH0)-R\1) 

By  solving  the  linear  equation  on  <3),  <22-  can  easily  determine  the  <5],  <22  ^om  the 
autocorrelation R{0),  /?(!),  and ^(2).  Equations  5.83  and  5.84  can  be  wrinen  in  vector 
form  as 


1?2-  a  =-r. 


where 


R7  = 


R(0)  Rm 
R(l)  R(0) 


a=[dua2\\  r^[Ril),Ri2)]'. 

therefore 

Thus  i=  [ai.^a]'  can  be  determined  from  r  *  [/?(l),/f(2)]'  and 


8=--^. 

R2 


R(0)  RiO) 
Ril)  Rm 


namely,  from/?(0),/?(l)and^(2) . 

After  <2(,  <22  are  obtained,  <7^^  can  be  estimated  from  Eq.  5.82, 

ai  =  Rm  +  aiR(l)~a2Ri2). 


(5.104) 

(5.105) 

(5.106) 

(5.107) 


(5.108) 


I  523.6  Spectrum  ofAR(2).  From  Eq.  5.59  foi  AR(2) 

X,  +  <2  iX|L-i  +  02X1-2  =  e, , 

I  X,^  +  aiX,_-i  +  02^»-r-2  =  .  (5.109) 

Taking  the  product  of  both  sides  of  these  two  equations,  respectively,  and  then  taking 
I  expectation  of  each  term, 

I  £[X,  •  X,_,  ]  +  £[X,_i  •  X,^_i]  +  a|  £[X^2  •  X^.2] 

I  +  <2 1  £1X,  •  X^-r-i]  +  a  102  +  02  £[X,  X^^_2  ] 

+  O1  E\X.t-r  X^_i]+Oi02  £[X»_/._iX/_2]  +  O2  £[X/_rX;_2]  =  £[6/ •€,_,]. 

1(5.110) 

Therefore,  for  r  0, 

I 


127 


/?^r){  1  +  af  +  +  ai[RJ^r  +  1)  +  RJ,r  -  1 )]  +  aia2[RAr  +  1 )  +  Rj,(r  -  1 )] 

+  a2[RAr  +  2)  +  RJ,r-2)]=Rf  (r)  (5.110’) 

According  to  the  relation  between  autocovariance  and  the  spectrum, 

—  ^RAr  T  l)e-^'’  =  —  '^RAr  =F  D  .  (5.111) 

-tTT  “tff 

Then,  transforming  both  sides  of  Eq.  5.110’  by  Fourier  transformation  and,  taking  into 
account  Eq.  5.111, 


Sx((0) 


1  +  af  +  02  +  (ai  +  aia2)(e"^  +  e*")  +  a2(e  ) 

1  7 

=  SeioD)  =  —at. 


Sx(a))\ 


2ji 


I  +  aie-^  +  a2e-^ 


2  2 
=  T~^e  • 
2ji 


Thus 


sM  - 


ai 


2jt 


-ioi  J. 


1  +aie  +  a2e 


(5.112) 

(5.112’) 

(5.113) 


from  the  function  a(Z)  of  Eq.  5.74 

sA(o)  = 


ai 


2jt 


(5.114) 


or 


oi 


2ji  |(1  +aia2  +  2(ai  +  aia2)costo  +  2a2Cos2m) 


oi 


2jt  (1  -a2)^  +  ai  +  2ai(l  +a2)coscu  +4a2Cosct)) 
Substituting  ai  forai  in  Eq.  5.88  gives 

SA(o)  - 


(5.115) 


(l-a2)[(l+fl2)^-ai2]ai 


2jr(l  +a2)|(l  -a2)^  +  ai+2ai(l  +a2)  costa  +4a2Cos^ta| 
ds(o)) 

From  this  expression,  and  settmg - =  0, 

da) 


(5.115’) 
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when  02  >0  and 


ai(l  +(32) 


4^2 


<  1,  the  spectnun  has  a  peak  at 


(Oo  =  cos' 


a, (I  +02) 


4<jj 


When  (32  <  0  and 
Fig.  5.19. 


ai(l  +02) 


402 


(5.116) 


<  1,  the  spectrum  has  a  trough  at  the  same  too  as  shown  in 


Fig.  5.19,  S(a>)  of  AR(2). 

52J.7  Example  ofAR(2).  In  Fig.  5.20,  an  example  of  the  AR(2)  process  for  oi  = 
-0.5, 02  =  0.8, =  1 .0,  expressed  as  X,  -  0.5  X^i  +  0.8  X,_2  =  e, ,  was  generated  over  r  =  1 

to  600,  using  the  same  pure  random  process  e,  as  was  generated  in  Fig.  5.4.  A  part  of  this 
process  (r  -  100  to  250)  is  also  shown  with  the  time  axis  extended.  Its  readings  are  listed 
in  Appendix  A1  as  Table  A1.3;  pp.  251, 254,  and  255. 


Figure  5.21a  shows  the  theoretical  autocorrelation  coefficient  p(r)  =  R{r)/R(0)  of 
this  generated  AR(2)  process,  calculated  by  Eq.  5.90,  using  the  values  of  coefficients 

ai  =  -0.5. 02  =  0.8,  and  the  varianceOf  =  1.0.  Part  b  of  the  same  figure  shows  the 

estimated  p(r)  =  ^(r)/^(0),  actually  calculated  from  the  generated  process.  AnAR(n) 
model  was  fitted  and  its  order  was  searched  by  AIC  criteria  and  found  to  be  n  =  2.  Then 
parameters  were  estimated  by  the  method  described  in  this  section  by  Eqs.  5.103’  and 
5.108.  The  results  were 

oi  =-0.54156,  02  =  0.81838,  and <7^=  1.04702, 

which  are  close  to  the  values  actually  used  in  generating  the  process.  The  autocorrelation 
coefficients  were  estimated  from  Eq.  5.90  using  these  values  just  as  were  the  theoretical 
values.  The  results  were  quite  similar  and  very  close  to  the  theoretical  values  in 
Fig.  5.21a,  so  again  the  drawings  were  omitted. 


Rg.  5.21a.  Theoretical. 

Fig.  5.21.  Autocorrelation  coefficient  for  AR(2)  process 
X,-0.5  +0.8  X«  =  e,.  e,:  NIO.  1], 
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Figure  5.22  shows  the  spectra,  Fig.  5.22a  the  theoretical  one  given  by  Eq.  5.115, 
Fig.  5.22b  the  estimated  one  from  the  fitted  model  AR(2),  and  Fig.  5.22c  the  one  esti¬ 
mated  by  the  nonparametric  method  from  the  generated  process  (Fig.  5.20),  using  the 
same  maximum  lag  number  (60)  and  Hanning  window  as  for  Fig.  5.5c  and  Fig.  5.12c. 
The  a  and  b  spectra  look  very  similar  to  each  other  and  in  this  case  c  also  looks  similar  to 
a  and  b.  In  Fig.  5.22c,  the  90%  level  of  confidence  interval  of  this  estimate  based  on 

the -distribution  of  the  equivalent  degree  of  freedom  (here  approximately  equal  to  27) 
as  is  explained  in  Section  2.5.5  is  shown  by  vertical  lines  for  reference. 
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Pig.  5.22c.  Estimated  by  nonparametric  method. 
Hg.  5.22.  (Continued) 


52.4  Second  Order  A utoregressive  F irsr  Order  Moving  Average 
Process,  A/?A£A(2.1) 

More  generally  for  AR(2),  when  the  residual  series  e’,  ofX,,  obtained  by  subtracting 
the  part  which  is  linearly  dependent  on  A;_i, 

€;  =  Xt-a'iX^i  (5.117) 

was  conelated  not  only  with  Xt.2  as  for  AR(2)  but  also  withe^i  as  shown  in  Fig.  5.23. 


(a)  et'vs.  X,_2.  (b)  c,'vs.  Cm 

Fig.  5.23.  Characters  of  e!  for  ARMA(2.1). 

Then 


€,'=  02  X,_2  +  bl€,_i  +  €,  .  (5.118) 

I 

I 
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(5.119) 


€,  is  now  a  purely  uncorrelated  random  process. 


£[€,■€  ^r] 


From  Eqs.  5.117  and  5.118 


r  =  0 
r^O' 


Xi-a'i  X,_i-<22  ■Xf_2-  t>i€,,i  +  €, 


this  equation  is  expressed  generally,  changing  -ai  toflj  and- <22’  loui.  as 

Xt  +  OiX-i  +  ^2X1-2  ~  ^1  ■  (5.120) 

When  the  process  X  is  expressed  by  Eq.  5.120,  where  a\,  02,  and  hi  are  constants  and 
€t  is  a  random  variable,  this  process  X,  is  called  a  second  order  autoregressive  first  order 
moving  average  process,  ARMA(2.1). 

Equation  5.120  can  be  written  as 

€,  =  X,  +  aiX,^i  +^22^,_2-hie,_i .  (5.120’) 

This  expression  shows  that,  in  order  to  compute  the  present  value  of  e, ,  we  need  e,_i ,  and 
when  we  compute  c,  recursively,  we  need  the  preceding  values  of  e^-i  starting  from  the 
beginning  e,_2 ,  e,_3  •  •  • .  This  situation  is  different  from  AR(1)  and  AR(2)  as  shown  by 

Eq.  5.7  and  Eq.  5.59  for  which  we  do  not  need  any  preceding  values  of  c^.i .  This  makes 
the  estimation  of  the  ARMA(2.1)  model  much  more  difficult  than  that  of  the  AR(1)  and 
AR(2)  models.  This  difficulty  can  also  be  shown  as  follows. 

From  Eq.  5.120’ 

€,_i  =  X,_i  +  aiX,_2  +  O2X1-S  ~  bi€i-2  •  (5.121) 

Inserting  this  into  Eq.  5.120  gives 

X,  =  -aiXi^i  -  aiXf^i  +  ^i|Xr_i  +  aiX,_2  +  a2X,L.3  -  hie,_2|  +  e* 


X,  =  (-«!  +  bi)Xi.i  +  (- 02  +  a\bi)  X,_2  +  fl2^lX,-3  -  bie,-2  +  e, .  (5.122) 

Here  6^2  is  still  included,  so  it  should  be  expressed  in  the  same  way  as  Eq.  5.121  by  X-2< 

X-3-  •  •  andei^3,andsoon.  However,  even  with  Eq.  5.122,  when  the  dependence  of  A) 
is  expressed  in  terms  of  past  X,  the  equation  is  nonlinear  in  the  unknown  parameters  O], 
02,  and  h].  As  a  result,  the  regression  becomes  nonlinear  and  requires  a  nonlinear  least 
squares  method  for  estimation,  which  is  different  from  the  case  of  AR(1 ),  AR(2)  or  the 
general  AR  model  as  will  be  shown  in  Section  5.4,  where  all  coefficients  can  be  obtained 
by  the  linear  least  squares  method.  The  estimation  of  oi.  02,  b\  for  ARMA(2.1 )  will  be 
shown  later. 

52.4.1  Green’s  Function  for  ARMA(2.1 ).  Equation  5.120  is  expressed,  using  the 
backward  shift  operator  B,  as 

(l+aiB  +  a2B^)X,=  {l+biB)e,.  (5.123) 
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Then 


X  =_JilML_g 

(1  +  Q\B  +  £l2B^) 
(1+biB)  ^ 


V  4 

1 

(l+fe)  1 

1 

(1  -fiiB) 

(>-£)  >-H 

>€, 


-S' 

>=o 


+  1  j  .  I  f^2  +  bi\  j\ 

-  1/^1  +  I  -  2^/-; 


=  X  ^'-j- 

j=Q 


(5.124) 


Green’s  function  is 


Gy=|  — )/.*(- 
'  -fi2  J  \fi2 


+  fel\ 


(5.125) 


When  hi  =  0,  then  Green’s  function  will  be  that  of  the  AR(2)  already  given  as  Eq.  5.70, 

Green’s  function  can  also  be  easily  derived  as  a  solution  of  the  homogeneous  equation, 
Eq.  5.120, 

( 1  +  +  a2B^)Xt  —  0 


(5.126) 


with  the  initial  conditions  as  follows. 
Substituting 


X.^lGj  Ch  = 

y.o 


S  °J  ^ 
0=0 


into  Eq.  5.123  gives 

/»  ^ 

OP 

{l+aiB+a2B^)  ^  Gj  B^  e,  =  (1  +  hiB)€, 

j=o 

(1  +  aiB  +  a2B^){,Go  + GiB  +  G2B^ +  •¥  •  •  •)€f  =  (1 +  hifi)e(.  (5.127) 


Comparing  the  coefficients  of  B  on  both  sides  of  Eq.  5.127  shows 
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Go  =  1 
Gqoi  +  Gj  = 

Gofl2  +  oiGi  +  G2  =  0 


Go=l  (5-128) 

Gi  =  bi-ai  (5.129) 

G2  =  -aiibi-ai)-a2  (5.130) 

Gj  =  -aiG^i-a2Gj-2,  j  ^  2.  (5.131) 


Eqs.  5.128  -  5.130  are  the  initial  conditions. 

The  solution  of  the  homogeneous  Eq.  5.126  is  the  solution  of  an  nth  order  differen¬ 
tial  equation,  that  is,  a  linear  combination  of  exponential  functions. /<•',  fi  being  the 
characteristic  roots,  i.e.,  the  roots  of  the  characteristic  equation /(Z).  Equation  5.73  in  this 
case  is  of  second  order. 

Therefore  for  this  case, 

Gj  =  gi  iu(  +  g2  Ml-  (5.132) 


Using  the  initial  conditions  given  by  Eqs.  5.128  and  5.129, 

+  52  =  1 

g\Mi  +  g2M2=Mi+M2  +  bi, 


where 

Solving  gives 


(jii+M2=-ai). 


51  = 


Mi  +  h 
Ml -M2 


(5.133) 

(5.134) 


(5.135) 


52  = 


M2  +  bi 

M2- Ml 


(5.136) 


Inserting  these  values  into  Eq.  5.132,  gives  the  same  expression  of  Green's  function  Eq. 
5.125  for  ARMA(2.1).  As  will  be  discussed  in  some  detail  in  the  next  section,  in  order 
that  the  original  process  X,,  which  can  be  considered  as  an  output  of  the  input  6, ,  be  sta¬ 
tionary,  M\  Ml  should  be  less  than  1  in  absolute  value,  I  I  <  1,  l|/2 1  <  1<  wbere^  \,Mi 
are  the  roots  of  the  characteristic  Eq.  5.73. 

y(Z)  =  -I-  aiZ  +  02  =  0 


Mh  M2 


_  -a\  Joi-^i 


(5.137) 


for  M\  -I’M!  - -o\  and  MiMi  -  oi-  When  a^-Aoz  <  0,  then  Mu  /*2  are  complex  and  con¬ 
jugate  to  each  other  as 


MuMi 


~ai  ,  .  jAa2-ai 

- ±  i - - 

2  2 


(5.137’) 
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When  this  was  expressed  as 


(5.138) 


y  =  luil  =  \H2\ 


■{(t) 


2  \jAa2-al 


1/2 


=  ^02 


,  y4<J2-<2l  _l  -^1 

o)  =  tan  ^ - = - -=  cos  ‘ ^  =  cos  ^  -  ■ 

2  >102 


~ax 

Then  inserting  Eq.  5.138  into  Eq.  5.125  gives 


f^2  +  bi 


(y  e-^)J 


yJ 


0^1  -/^2) 

^yJ. 


[(fix  +  (^2  +  bx)e-j^‘^^] 


Therefore,  if  putting 


ai-2hi 

cos  JO) — 7=--- . — . sm  JO) 

—  ax  —  2bx 


^Aa2-ax 


=  C/, 


Gj  =  y  V 1  +  C/  ^  cos(/to  +  d) 


where 


5  =  tan  ^ 


at -2^1 

Jaot^x 


Now  let  Jl  +  lfi  -  V  again, 


Then 


Gj  =  yjV  COSijO)  +6) 

=  Vc>‘®c^y>+ye>'®e-^y-' 

=  .?i  y-'+^2  y-'- 

gi,52  =  =  |g|e± + <5) 


(5.139) 

(5.140) 


(5.141) 

(5.142) 

(5.143) 

(5.144) 


(5.145) 

(5.146) 
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Therefore  (5.147) 

Arg[^]=>a)+<5.  (5.148) 

From  Eq.  5.143  Gj  caa  be  expressed  as  a  damping  cosine  curve  with  damping  y  = 
determined  by  uo  aitd  the  period  by 

2jr  2tc 

TT'cos-"^' 

2  va2 

After  02  was  fixed,  period  is  decided  by  a\  and  phase  that  is  d  different  from  that  of 

•  1^1  —  2Z)i 

by  o  ss  tan~‘  ,  ,  as  is  shown  in  Fig.  5.24. 

yj 402-0^ 


Green’s  function  and  the  autocovariance  function  are  closely  related  as  expected,  because 
using  Green’s  fimction, 

Rir)  =  £[Xi  X^] 


=  (5.149) 

7=0 

52.42  Inverse  Funaion  ofARMA(2.1).  InEq.  5.123 

(1  +  flj  5  +  a2B^yXi  =  (1  +  bj  B)  Cf . 
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If  we  express  e,  by  an  inverse  function  as 
00 

€/  =  ^  -Ij  X,^j  =  {-lo-hB ■  ■  ■  Ij  B^  ■■  ■)X(, 

then  Eq.  5.123  becomes  (1  +a]B  +  a2B~)Xt  ~  (1  +b\B)(-IiB  -hB' - Ij  B^  ■  ■  ■)A’,. 

Cmnparing  the  coefficients  of  the  powers  of  B  gives 

-/o=l 
ai  =  bi -h 


02  —  —b\I\  —Iz 
0  =  —  /s  —  b\l2 

0  =  -Ij-bilj-\j^i 


(5.151) 


or 

11  =  bi-ai 

12  =  -a2-bi(bi-ai)  (5.151’) 

//■  ~  ~  bilj-.i^jzi- 

The  last  equation  of  Eq.  5.151’  is  (I  +  b\B)  Ij  =0  for  j  s  3. 

Therefore  Jj  —  ~bilj-i.  (5.152) 


This  Eq.  5.152  is  the  recursive  equation  for  Ij. 

52.4J  Stationaliry  and  Invenibility  ofARMA(2.J).  The  statiocality  of  ARMA(2.1) 
depends  on  the  convergence  of  the  free  oscillation  that  comes  from  the  complementary 
function  or  the  solution  of  the  homogeneous  equation.  The  homogeneous  equation  of 
ARMA(2.1)  is  the  same  as  the  homogeneous  equation  of  AR(2),  5.60,  which  is 

Xt  +  oiXt-i  +  ozXfjz  ~  0. 

Accordingly,  the  relations  required  of  the  coefficients  oi  and  az  for  the  ARMA(2.1)  pro¬ 
cess  to  be  stationary  are  the  same  as  for  AR(2),  as  has  already  been  shown  in  Section 
5.2.3  and  in  Figs.  5.15  and  5.16. 

On  the  other  hand,  the  invenibility  of  ARMA(2.1)  depends  on  the  boundedness 
00 

of  €i,€t  =  j  »  and  from  the  relation  derived  as  Eq.  5.152, 

Ij  =  -b\Ij^\,  Ibl  should  L  ?  less  than  1 .  This  requirement  corresponds  to  the  character 
required  for  ai  of  AR(1),  Util  <  1,  so  that  the  process  AR(1)  is  stationary  when 
Gj  =  ai  Gj^  1. 

52.4.4  Autocovariance  Function  for  ARMA(2.1).  From  Eq.  5.120 

Xt  =  ~0lXt-i—02Xt-2  +  + 

Therefore,  multiplying  both  sides  by  and  taking  the  expected  values. 
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E\X,  •  X,^]  =  -  axE[X,_^  ■  X^r\  -  ^2  E[  ^,-2  ' 

+  £[e,  ■  Xi.r]  +  biE[€  ,_i  .  (5.153) 

Anotber  way,  by  Green’s  function, 

00 

Xt=  ^  Gj  e^.)  =  Go  €,  +  Gi  £,_i  +  G2  f<_2+-  ■  Gj  €i-j  +  -  ■  ■ 

;  =  0 

where  Gj  is  expressed  by  Eqs.  5.128  to  5.131.  Accordingly, 

X,_r=  Go  e,_^+Gie,__i  +  G2€^.^_2+ •  •  •Gj€,^r-j+-  ■  ■ 

=  €/_/-+ (^1 -ai)€|_r-l  +  {-<32“^i(^1  f;-r-2+'  '  ' 

+  {-a2G^2-^lG^l|  €,_r_;+-  •  (5.154) 

Using  these  relations  from  Eq.  5.153  gives 

r  =  0  when  /?(0)  = -ai/?(l)-a2'R(2)  +  af  +  (bi -fli)  biO^  (5.155) 

r=l  when /?(1)  = -ai/?(0)- <22^(1)  +  (5.156) 

r  when/?(r)  =  -ai£(r-l)-a2£(f-2).  (5.157) 


In  order  to  express  the  values  of  R(r)  recursively,  we  have  to  solve  for  RiO)  and 
/?(!),  using  Eqs.  5.155  and  5.156  after  substituting  R(2)  from  Eq.  5.157.  Then  Eqs.  5.155 
and  5.156  will  be 


(1  -a|)  i?(0)  +  fli(l  -a2)/?(l)  =  (1  -aibi  +  bf)  a}  (5.158) 

fli£(0)  +  (l +a2)^(l)  =  (5.159) 


Therefore 


£(0)  = 


{l-aibi  +  bl)  aiil-az) 
bi  (l+az) 


(l-a|)  fli(l-a2) 
ai  (l+fl2) 


oi. 


(5.160) 


£(!)  = 


(l-a|)  il-aibi  +  b^) 
ai  bi 


(l-a|)  Old -02) 

Ol  (1+02) 

R(r)  =  -  aiRir  - 1 )  -  azRir  -  2) 


a 


2 

e> 


r  S  2, 


(5.161) 


(5.162) 
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Q{0)  =  1 


(5.163) 


e>(i)  = 


(l-Oo)  (l-aibi  +  bf) 

(^i)  (Bi) 

{l-aibi  +  bl)  01(1-02) 

(bi)  (1+^2) 


g(r)  =  -0iQ(r-l)-02g(r-2)  rS 


2. 


(5.164) 


(5.165) 


These  are  the  autocovariance  functions  of  ARMA(2.1).  As  was  done  for  AR(2)  in 
Section  5.2.3,  we  can  get  the  same  results  as  the  solution  of  the  homogeneous  equation 
because  Eq.  5.157  is  the  homogeneous  equation  with  the  same  coefficient  as  the  homoge¬ 
neous  Eq.  5.126  or  Eq.  5.60  of  AR(2)  expressed  by  Eq.  5.59. 

Its  solution  is  in  the  shape  of 

R(r)  =  Bi  fil+B2  (5.166) 

being  the  roots  of  characteristic  equation 

f(Z)  =  Z^  +  OiZ  +  O2  =  0  (5.167) 

or 


Hi+fi2  =  -0\ 
fllH2  =  02. 


B 1  and  B2  are  constants  and  are  determined  by  the  boimdary  conditions 

5 1  +  B2  —  R(^) 

Bifii  +  B2M2  —  B(l). 


After  algebraic  manipulation, 

=  aigi  *1 


82 


(1  -fif)  (1  -f2i  H2) 


Bi  =  (^€82 


82 


81 


(5.167) 


or,  using  the  relations  of  Eqs.  5.135  and  5.136, 


2C“i  +  &i) 
Bi-Of—  — 2 


{lll-fl2) 


(fi\-\-b\)  /i2  +  bi 


(I -fit)  l-fl\fl2 


(5.168) 
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r 


B2 


fi2  +  b\  fil  +  bi 


1  -ni  1 


and 


/?(0)  =  a2j 


=  a}- 


8i 


82 


2gi  g2 
i-fif  1  -fii  1  -fnfi2 


(Mx  +  biY  ^  Ui2-^bf)  2{pii^bx){ji2-^bx) 

{I -fix)  (1-/^?)  ^-fi\ft2 


(5.169) 


(5.170) 


R{l)  =  a}- 


Ai 


+  al 


(fix -fi2) 
fi2 


ifix-fii) 


{fix^bxf  (fix-x-bx){fi2  +  bx) 
(1  -fi\)  1  -fiiMi 


(fi2  +  bxr  iMx  +  bx)(ji2  +  bx) 


(I-A2) 


1  -fixfl2 


(5.171) 


Using  the  B\,  B2  in  Eq.  5.166,  we  get  the  general  expression  for/?(r). 

The  autocovariance  function  can  be  expressed  using  Green’s  function  as  Eq.  5.149. 
Then,  starting  from  the  expression  of  Green's  function 

<^J  =  8i  Mi+82Ml 

(he  same  results  can  be  derived  for  /?(r),  although  the  manipulations  will  not  be  shown 
here. 

52.4J  Estimations  ofa\,  a^,  b\,  anda}  <rfARMA(2.I).  From  Eq.  5.157 
It(r)  =  -axRir-l)-a2R(r-2)  r  S  2. 

Therefore,  for  estimation,  with  the  sample  values  of  autocovariances, 

M2)  =  -diMl)-MM0)  (5.172) 


f?(3)  =  -aii?(2)- 02^(1). 


(5.173) 
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Solving  these  linear  equations  gives 


A 

ai  = 


k2)  -R(0) 
k3)  -^(1) 

i-Rd)  -/?(0) 


1-^(2)  -Ra)\ 


R(3)R(0)-kim2)  gO)-gd)g(2) 
R\l)-kO)Ri2)  *  r(l)-e(2) 


(5.174) 


-^(1)  ^(2) 

i-J?(2)  ^(3) 

-ki)  -ko) 

-R(2)  -kl) 


k(2)-kl)RO)  gH2)-Qd)Q{3) 
RHl)-kO)Ri2)~  QHl)-g(2) 


(5.175) 


After  j]  and  62  have  been  determined  bi  and <7/  can  be  obtained  by  solving  Eqs. 
5.155  and  5.156.  From  them,  Pandit  and  showed  that  bi  can  be  obtained  as  the 
solution  of  a  quadratic  equation 


bl  +  Cbi+l  =  0 


(5.176) 


where 


C  —  — Qj  + 


«  A  A  A  A 

l+aiQi+aTgi 

-01-02- l)Qi 


(5.177) 


although  this  is  quadratic  in  terms  of  b\  and  is  not  a  linear  relation. 

52.4.6  Spectrum  ofARMA(2.1).  The  expression  of  ARMA(2.1)  is,  by  Eq.  5.120, 

Xt  +  a\X,..\  +  =  €/  +  ^1  ff-i 

or,  using  the  backward  shift  operator  5,  by  Eq.  5.123, 

a(B)X,=fiiB)€,  ^5  J20’) 

a(Z)  =  1  +  aiZ  +  a2Z 
'  /?(Z)=  I+61Z. 

Therefore 

X(-r  +  f »-r+  f-r-l  •  (5.178) 


Taking  the  product  of  Eqs.  5.120  and  5.178  and  then  taking  expected  values  of  each  term 
gives 
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E[X,  •  X,.r]  +  ajE[X,.i  ■  +  a^[X^2  ■ 

+  a\E\Xt  ■  +  aia2E[X,^iX^^2]  +  02[X,  ■  X,_r-2\ 

+  aiE[Xt.r  +  aia2E\X,^_\  •  X,_2]  +  a2E[X,^r  ■  ■X/_2] 
=  £[e,-e,_r]  +  fti£[€,_i-  ^#-r— 1 

+  fci£[e,_i  •  6;-,-], 


£;,(r) 


1 1  +  + a|j  +  +  1 )  +  RJj  - 1)]  +  a\a2[Rxi.r  +  1 )  +  R:Jj  - 1 )] 

+  +  2)  +  Rj^r  -  2)] 

=  /2f(r)[l  +  f)i2j  +  i,i[/2^(r+l)  +  £,(r-l)].  (5. 

Here  we  use  the  relation  shown  by  Eq.  5.111 


—  y£(r  T  1)^-^"  =  — y£(r  T 

2jr^  2jc"^ 


-iff 


-iff 


=  s{o})  • 

Taking  the  Fourier  transform  of  both  sides  of  Eq.  5.179  gives 

^;e(ty  )|  1  +  oi  +  +  02i.e~^  + 

=  ^<T2jl+b?+hi(e‘“  +  e-^)). 


sA(o) 


1  +  aie-^  +  ^2  2  =  ^a}  |l  +  bie-^\  ^ . 

2jt 


Therefore 


sA(o)  = 


2?r  |a(e-^)p 


(5. 

(5. 

(5. 


1+ 

2 

~  2jc 

l+fli  e-^  +  a2e-^\ 

2 

a}  {l+di2+2hi(e-^  +  e'“')) 

2^  |l  +  af  +  <72  +  (^1  +  <3'i02)(^~*“  +  +  02ie~^  +  e^)j 

fff  I  +  bi  +  2b  icoso) 

2ji  [l+ai  +  a2  +  2ai(l+a2)cos(o  +  2a2  cos2a>  ] 


(5. 


Here  A/  is  assumed  to  be  1,  as  was  mentioned  at  the  beginning  of  Part  n.  As  a 
result,  5jt(<y)  are  obtained  in  the  range  -7t<w  <n.  However,  if  Ar  5^  1  and  the 
spectrum  is  calculated  for  -  jr/Ar  <  o  <  jr/Ar  ,  then  the  spectrum  is  easily  inverted  into 


sAto)  -  Ar  ■ 


(5.1851 


52.4.7  Example  ofARMA(2.1)  Simulated.  Figure  5.25  is  an  example  of  a  gener¬ 
ated  ARMA(2. 1 )  process  over  r  =  1  to  600,  when  a  =  -0.3,  <71  =  +  0.4,  bi  =  -  0.7,  and 

Cf  =  1.0,  namely  by 


;s:,-o.3  x,.i  +  0.4  x^2  =  ft- 0.7  f,_i . 

Here  €/  is  the  pure  random  process  we  generated  as  AR(0)  in  Fig.  5.3.  Its  readings 
are  listed  in  Appendix  A1  as  Table  A1.4;  pp.  251, 254,  and  255.  The  correlations  and 
spectra  of  this  example  are  shown  in  Figs.  5.26  and  5.27.  Figure  5.26a  shows  the  theoreti¬ 
cal  autocorrelation  coefficient  g(r)  -  R(r)/R(0)  of  this  generated  ARMA(2.1),  calcu¬ 
lated  by  Eqs.  5.160, 5.161, 5.162,  or  5.163, 5.164, 5.165,  using  the  design  values 
of  =  -  0.3,  a2  =  0.4, 6]  =  -  0.7  anda^  =  1.0.  Figure  5.26b  shows  the  estimated 

g(r)  =  R(r)/R(0),  calculated  from  the  generated  process. 

We  can  see  in  Fig.  5.26b  that  the  autocorrelation  coefficient  at  higher  lag  r  continues 
to  fluctuate,  even  after  the  theoretical  coefficient,  shown  in  Fig.  5.26a,  has  died  down  and 
converged  to  zero.  On  the  other  hand,  the  values  at  low  lag  numbers  look  very  similar  to 
the  theoretical  values.  An  ARMA(n,m)  process  was  fitted  and  its  optimum  orders  n  and 
m  were  searched  by  AIC  criteria  as  explained  in  Section  5.5.4,  using  these  autocorrela¬ 
tions  b,  and  actually  were  found  tobens2,  m  =  l.  Then  the  parameters  were  estimated 
by  the  method  described  in  this  section  as  Eqs.  5.174, 5.175, 5.155,  and  5.156.  The  values 
thus  obtained  are 

=-0.37842,  <72  =  0.40130,  =-0.73968,  0^=1.0400 

which  are  not  very  close,  but  pretty  close  to  the  values  actually  used  to  generate  this 
process.  The  autocorrelation  coefficients  g(r)  -  R(r)/R(0)  were  then  estimated  using 
these  estimated  parameters  by  Eqs.  5.160, 5.161,  and  5.162,  just  as  for  the  theoretical 
values.  The  results  are  similar  and  very  close  to  the  theoretical  values  shown  in 
Fig.  5.26a,  so  that  drawings  were  again  omitted. 
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Figure  5.27  shows  the  spectra,  the  theoretical  one  from  Eq.  5.182”  in  Fig.  5.27a,  the  one 
estimated  from  the  fitted  model  ARMA(2.1),  using  the  estimated  parameters  in  Fig. 
5.27b,  and  the  one  estimated  by  the  nonparametric  method  from  the  generated  process  of 
Fig.  5.25  in  Fig.  5.27c,  with  a  maximum  lag  number  of  60  and  Hanning  window.  Spectra 
a  and  b  are  similar  and  close  to  each  other,  and  c  fluctuated  as  expected.  In  another  trial, 
instead  of  the  ARMA  (n,m)  model,  an  AR  model  was  fitted  to  this  generated  ARMA(2.1 ) 
process.  The  optimum  order  n  was  searched  by  AIC  criteria  as  AR(n),  and  was  found  to 

be  /»  =  6.  Its  parameters  Si  to  S^  and  were  estimated  by  the  method  described  in 
Sections  5.4.3  and  5.4.4  as 

<Zi  =-0.35801,  ^2  =  -0.64448,  ^3  =  - 0.45424, 

^4  = -0.34915,  ^5  =  -0.10554,  4  = -0.1 1502, 
ai  =  1.03499. 

These  values  are  listed  in  Table  5.1. 

The  value  of  a}  is  very  close  to  the  theoretical  value  of  =  1.0.  The  spectrum 
of  this  fitted  AR(6)  is  shown  as  Fig.  5.27d.  It  is  interesting  to  find  that  this  spectrum 
Fig.  5.27d  is  very  smooth,  but  its  shape  looks  like  the  smoother  shape  of  spectrum 
Fig.  5.27c,  the  one  estimated  by  the  nonparametric  method.  In  Fig.  5.27c,  by  vertical 

lines,  the  90%  level  of  confidence  interval  of  this  estimate,  based  on  the  ^'-disnibution 
of  the  equivalent  degree  of  freedom  (here  approximately  equal  to  27)  as  is  explained  in 
Section  2.5.5  is  shown  for  reference. 

As  another  example  of  the  ARMA(2.1)  process,  the  sign  of  the  parameter  b]  in  the 
previous  example  was  changed,  and  another  ARMA(2.1)  process  was  generated,  keeping 
ai,  02,  anda^^  the  same, 

ai  =  -0.3,  02  =  +  0.4,  61  =  +  0.7,  ai  =  1.0. 

As  for  the  previous  example,  its  time  history  (readings  are  listed  in  Appendix  AI  as  Table 
A1.5;  pp.  251, 256,  and  257),  autocorrelation,  and  spectra  are  shown  as  Figs.  5.28, 5.29, 
and  5.30. 

In  this  example,  the  AIC  criteria  gave  ARMA(3.1)  as  the  optimum  model  to  be 
fitted,  instead  of  ARMA(2.1),  though  the  difference  in  AIC  is  not  so  large.  The  estimated 
spectrum  of  this  optimum  ARMA(3.1)  is  shown  as  Fig.  5.30b,  and  for  reference  the 
spectrum  of  the  fitted  ARMA(2.1)  model  is  shown  as  Fig.  5.30e.  ARMA(3.1)  gave  the 
minimum  value  of  AIC  and  ARMA(2.1)  did  not.  However,  the  ARMA(2.1)  spectrum  Fig. 
5.30e  looks  more  like  the  theoretical  spectrum  Fig.  S.30a  than  does  that  of  ARMA(3.1), 
spectrum  Fig.  5.30b.  Spectrum  Fig.  5.30d  shows  the  spectrum  of  the  fitted  optimum 
AR(n)  model,  where  n  appeared  to  be  8  by  AIC  criteria.  It  is  interesting  to  find  that  this 
spectrum  Fig.  5.30d  shows  again  the  same  pattern  of  variation  as  spectrum  Fig.  5.30c 
obtained  by  the  nonparametric  method.  The  estimated  values  of  parameters 

01,02,-  ■  ■  Os  andai  are  listed  in  Table  5.1,  and  di  =  1.0491 1  is  very  close  to  the 

theoretical  value  ai  =  1.0. 


Rg.  5.27a.  Theoretical  ARMA  process  ARMA(2.1 ). 


Fig.  5.27b.  Estimated  by  ARMA  model  fitting  as  optimum  ARMA(2.1). 

Fig.  5.27.  Spectrum  for  ARMA(2.1 )  process  (1 ) 

X,-  0.3  Xm  +  0.4  X«  =  € ,-  0.7€  m  ,  e  r :  N[0, 1  ]. 
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riTi! 


Fig.  5.29a.  Theoretical. 


Fig.  5.29b.  Estimated. 

Rg.  5.29.  Autocorrelation  coefficient  for  ARMA(2.1)  process  (2) 
X,-  0.3  X„  +  0.4  X«  =  € ,+  0.7e  m  .  e  r :  NIO,  1]. 


Rg.  5.30a.  Theoretical  Al 

Rg.  5.30.  Spectrum  for  ARM 
X,-0.3  X„+0.4 
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Tat^e  5.1 .  Results  of  estimation  by  model  fitting  —  [I]. 


Fig.  No. 

Process  ARMA  (n.m) 

Generated 

Readings 

in 

Estimated  as 
Optimum 

Estimated  Equivalent 
AR  (N) 

5.3 

AR  (0) 

■■ 

0 

A1.1 

0 

— 

5.5 

ARMA  (0.0) 

n 

1.04649 

5.10 

AR(1) 

n 

1 

1 

a 

A1.2 

-  0.50933 

— 

5.12 

ARMA  (0,1) 

o; 

mm 

1.04646 

520 

AR(2) 

■■ 

2 

2 

H 

-0.5 

A1.3 

-0.54156 

— 

4-0.8 

+  0.81838 

522 

ARMA  (0.2) 

wM 

1.0 

1.04702 

525 

ARMA  (2,1) 

n 

2 

2 

6 

m 

1 

1 

0 

[Process  (1)] 

-0.3 

-  0.37842 

a, -0.35801 

a. -0.64448 

+  0.4 

A1.4 

+  0.40130 

aj- 0.45424 
a*- 0.3491 5 

■ 

-0.7 

-0.73968 

a, -0.1 0554 
a«-0.11502 

527 

ARMA  (2,1) 

H 

1.0 

1.0400 

of:  1.03419 

528 

ARMA  (2,1) 

n 

2 

^  {  NOT  \ 

8 

m 

1 

1  1  OPTIMUM  1 

0 

(Process  (2)] 

-  0.34559 

a,  1.00947 

02 

+  0.41494 

aj- 1.08338 

b, 

+  0.69856 

aj  0.09614 

of 

1.0 

1.0558 

04-0.50333 

n 

2 

”  A  1.9 

3 

a,  0.40894 

m 

1 

^  (OPTIMUM) 

a,- 0.30626 

-  020995 

a^  0.17930 

02 

+  0.30378 

a, -0.09768 

Oi 

HBI 

+  0.15392 

b, 

+  0.80692 

5.30 

ARMA  (2,1) 

of 

1.0 

1.0440 

of:  1.04911 

5.30 


ARMA(2,1) 


1.0 


oi  1.04911 


52  J  ARMA(J.I).  MA(2)  ARMA(12)  MA(1).  andARMA(22) 

523.1  ARMA(1.1).  In  ARMA(2.1).  expressed  by  Eq.  5.120 

Xt  +  <21  X,_i  +  <22  A,-2  =  /-I  » 

when  02  =  0,  the  prcxess  is  called  ARMA(1 .1 ).  We  can  easily  get  Green's  function,  the 
inverse  function,  the  autocovariance,  and  the  spectrunt  function  of  ARMA(l.l)  by  modi¬ 
fying  those  functions  for  ARMA(2.1)  by  setting  02  =  0. 

5232  MA(2).  When  uj  =  <12  =  0  in  5.120  and  there  exist  hi  in  addition  to  b\, 

Xt  —  €/+  bi  €/_i  +  f>2  ff-2-  (5.184) 

This  process  is  called  MA(2). 

In  this  case  Green’s  function  is  the  function  for  expressing  X,  by  MA  ( 00 ).  so  to 
speak  of  Green’s  function  for  a  pure  moving  average  process  has  no  meaning.  However, 
formally, 

f=0 

and 

Go  =  1 

Gi  =  bi 

G2~b2 

Gj  =0  y  S  3. 

In  this  case  the  inverse  function  for  expressing  Xt  by  AR  ( »)  is  meaningful.  From  Eq. 
5.184, 


Xt  =  (1  +  biB  -t-  f  I 


(5.184’) 

here 

/3(Z)  =lAbiZA  biZl 

(5.185) 

Now  supposing  the  quadrant 

/2(Z)  =  Z2  -t-  biZAb2  =  0 

(5.186) 

has  two  rootsvi  andV2,  /2(Z)  =  (Z-vi)(Z-V2) 

*'1,V2=-^(-^71  ±  ylb\-Ab^  . 

(5.187) 
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Then 


jl  - (Vi  +  V2)B  +  Viva^^je,  =  X, 

V\->rV2  =  -bx 
Vi  V2  =  b2 

{1~vxB){\-V2B)  e,  =  X,. 


€,  = 


X,  ^  X,  1 

^(B)  (1 +  ~  (1-Vi5)(1-V25) 


1 


V1-V2 

1 

V1-V2 


Vl 


V2 


1  -ViB  1  -V2B 


X, 


1  (vr- 

U  =  o 


j+i  j+i\ 
vi  -vi 


V1-V2 


i-j- 


Therefore,  if  the  expression  of  the  inverse  function  /,  is  used  here  as 

00 

£,= 

1=0 


(5.188) 


Ij 


V 


1  ^2 
V1-V2 


Vl 

V2-V1 


Vl 


V2 

V]-V2 


A- 


(5.189) 


This  relation,  as  well  as  the  way  to  derive  it,  is  the  same  as  that  used  for  the  derivation  of 
Green’s  function  for  AR(2),  already  shown  in  Section  5.2.3.  For  invertibility  of  MA(2), 

€  t  must  be  boimded  by  t  -*  t-j,  j  -*■  » .  Then  by  the  same  logic  as  was  used  for  AR(2), 

lvil<l,  Iv2l<l  should  stand  for  MA(2),  and  the  conditions  that  must  be  satisfied  by 
b\,  b2  are  the  same  as  the  conditions  that  had  to  be  satisfied  by  a\,  02  of  AR(2).  This  con¬ 
dition  is  shown  in  Fig.  5.31,  which  is  similar  to  Figs.  5.15  and  5.16,  where  a\,  02  are 
replaced  by  bi,  bz-  This  triangular  zone  is  again  subdivided  into  subzones  [I]  and  [11], 
depending  on 


bl  ^  4b2[I]  and  bf  <  4b2[n]. 
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Subzone  [I]  is  divided  into  four  regions: 

0  bi<Q,b2>0 
<  0,  ^2  <  0 
bi  >  0,  &2  >  0 
bi  >  0,  />2  <  0 

as  shown  in  Fig.  5.31,  just  similar  to  Fig.  5.16. 

i  bj 


1.0 


Rg.  5.31.  Invertible  Subzones  [I]  and  [II];  0,  0,  0,  0  forbi,  bg. 

X,  -  e,+  bi€f_i+  bi^t-i 

Xf-r  =  €t^  +  bie^_i+  b2€i-r-2- 

Therefore,  multiplying  both  sides  of  these  two  equations  respectively  and  taking  the  ex 
pected  values  of  each  term, 

E\Xl  •  X(-r]  =  f  /-I  '  ^t-r-1  ]  +  ^2  ^[^1-2  ^t-r-2  ] 

+  bi|£[e^.i  ]  +  £[€■/  •  ]|  +  b2|£'[e  t~2  Ci-r  ]  +  £’[e,e,^_2]) 

+  bib2|£[e^.i  •  €,_r_2  ]  +  E[€,-2  ■  £i-r-l  ]|- 


because 

E[€t  fl-r] 


RAO)  =  a bl)  (0) 

(5.190) 

RA))=RA-\)  =  ibib2  +  bi  )R,  (0) 

(5.191) 

RA2)  =  RA~2)  =  b2R,iO) 

(5.192) 

RAr)  =  0  r  2:  3 

(5.193) 

r  =  0. 


dr  is  Kronecker’s  delta  function. 


RAr)  =  0  for  r  s;  3.  This  is  an  interesting  characteristic  of  MA(2).  Generally,  for 
MA(n),f?;f(r)  =  0  for  r  >  «  +  1.  In  the  same  way  as  for  ARMA(2.1) 


(5.194) 


Estimation  of  hi,  b^,  andCf  is  possible  by  solving  the  quadratic  Eqs.  5.190,  5.191,  and 
5.192  for  hi,  ha,  andffe . 

5253  Example  ofMA(2),  MA(1),  andARMA(22).  As  was  done  for  AR(0),  AR(1), 
AR(2),  and  ARMA(2.1)  in  the  preceding  sections,  an  MA(2)  model  was  generated  over 

/  =  1  to  600  by  .Y,  =  e,  +  0.2f  ji_i  +  0.i€u2  as  in  Fig.  5.32.  Fore/ ,  the  same  pure  random 
process  N[0, 1]  as  was  generated  in  Fig.  5.3  in  Section  5.2.1  for  AR(0)  is  used.  Its  read¬ 
ings  are  listed  in  Appendix  A1  as  Table  A1.6;  pp.  251, 256,  and  257.  Figure  5.33a  shows 

the  theoretical  R(r)  using  the  design  values  hj  »  +  0.2,  ha  =  +  0.8,  and  ai  =  1 .0,  and 
Fig.  5.33b  the  estimated  sample  autocorrelation ff(r).  It  is  interesting  to  note  that  theo¬ 
retically  f?(r)  =  0  for  r  S  3. 

Figure  5.34a  shows  the  theoretical  spectrum  given  by  Eq.  5.194  where  h]  =  +  0.2, 
ha  =  +  0.8,  anda^  =  1.0,  which  are  all  design  values,  and  Fig.  5.34b  the  spectrum  of  the 

fitted  model  given  by  Eq.  5.194  where  hj  =  0.18662,  ha  =  0.78000,  andOe  =  10816  calcu¬ 
lated  by  the  method  described  in  Section  5.3.3.  Of  course  again  in  this  example,  order 
determination  by  AIC,  to  be  explained  in  Section  5.5,  gave  the  proper  value  of  m  =  2. 
Figure  5.34c  is  the  spectrum  calculated  by  the  nonparametric  method.  Here  again  the 
90%  level  confidence  interval  is  shown  by  vertical  lines  for  reference.  This  was  originally 
generated  as  MA(2).  However,  as  will  be  discussed  in  Section  5.4,  often  an  AR  model  is 
fitted  generally  to  the  processes  taking  advantage  of  the  fact  that  the  determination  of 
parameters  is  much  easier  than  for  ARMA. 

Figure  5.34d  shows  the  spectrum  of  the  AR  model  fitted  to  this  process.  The  order 
of  this  process  is  «  =  13  and  is  naturally  higher  than  2  for  m  as  MA(m).  This  result 
shows  that,  if  this  MA(2)  is  to  be  approximated  by  AR(n),  AR(13)  is  the  closest  model  to 
be  adopted.  Again,  It  is  interesting  to  note  that  the  spectrum  Fig.  5.34d  of  AR(13)  looks 
very  similar  to  the  spectrum  Fig.  5.34c  estimated  by  the  nonparametric  method. 
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Rg.  5.33b.  Estimated. 

Rg.  5.33.  Autocorrelation  coefficient  for  MA(2)  process 
Xt  *  €  f  +  0.2  €  M  +  0.8  6  f.2 »  €  f .  NfO,  11. 
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Figures  5.37  and  5.40  are  the  spectra;  a  the  theorehcal,  b  that  estimated  b)’  model 
fitting  as  optimum,  c  that  estimated  by  the  nonparametric  method,  and  d  the  spectrum  of 
the  fitted  AR(/!)  model.  The  n  appeared  to  be  4  and  8  for  Fig.  5.37  and  Fig.  5.40, 
respectively. 


Fig.  5.37a.  Theoretical  MA(1 ). 


5  00-1  t  _  0.70294 


Fig.  5.37b.  Estimated  by  MA  model  fitting  as  optimum,  MA(1). 

Rg.  5.37.  Spectrum  of  MA(1 )  process  (1 ) 

X,  =  e,-0.7eM.  e,:N[0,1]. 
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Fig.  5.39b.  Estimated. 

Rg.  5.39.  Autocorrelation  coefficient  for  MA(1)  process  (2) 
X,  =  e,+  0.7eM.  c,:NI0,1]. 


A  A  o 

The  values  of  parameters  (and  i>2.  ai  if  any)  and  ofOe  estimated,  and  also  of 
estimated  a\  -  •  ■  a„  of  AR(n)  models  are  listed  m  Table  5.2.  By  comparing  Fig.  5.37  and 
Fig.  5.40,  we  can  find  the  difference  of  the  pattern  of  the  spectra  by  the  difference  of  the 
sign  of  parameter  b\.  The  same  tendency  appears  in  the  difference  between  Fig.  5.27  and 
Fig.  5.30.  We  can  recognize  this  difference  also  in  comparing  the  original  form  of  the 
generated  processes  in  Fig.  5.35  and  Fig.  5.38,  and  Fig.  5.25  and  Fig.  5.28. 

Figures  5.41, 5.42,  and  5.43  show  the  results  for  a  generated  ARMA(2.2)  model.  Its 
time  history  readings  are  listed  in  Appendix  A1  as  Table  A  1.9;  pp.  251  and  260.  The  fig¬ 
ures  are  the  same  as  for  the  other  examples,  so  no  explanation  wiU  be  given  here.  The 

A  ^  A  ^ 

estimated  values  of  parameters  a„,  b„,  anda^  are  listed  in  Table  5.2.  The  optimum  m,  n 
for  a  fitted  ARMA(/i,m)  model  appeared  to  be  n  =  2,  m  =  2  by  AIC  criteria  for  this  case 
too.  In  this  case  6}  appeared  to  be  0.84312,  quite  different  from  o}  =  1.0,  but  we  could 
not  find  the  reason  or  the  error  for  their  poor  estimation. 


Fig.  5.41.  Simulated  ARMA(2.2)  process 

X,-0.5X„+0.8Xw  =  e,+  0.2eM  +  0.8€«,  NtO,!]. 
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-»  (0 


Rg.  5.43a.  Theoretical  ARMA(2.2). 


Rg.  5.43b.  Estimated  by  ARMA  model  fitting  as  optimum,  ARMA(2.2). 

Rg.  5.43.  Spectrum  for  ARMA(2.2)  process 

Xf “ O.SXm  +  0.8X|^  =  €|+ 0.2c M  +  O-St  1.^ ,  Ct-  N[0, 1]. 
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Table  5.2.  Results  of  estimation  by  model  fitting  —  [II]. 


Fig.  No. 

Process  ARMA  (n.m) 

Generated 

Readings 

in 

Estimated  as 
Optimum 

Estimated  Equivalent 

AR  (N) 

5,?2 

MA  (2) 

n 

0 

0 

13 

m 

2 

2 

0 

0-  +0.374 

b, 

+  0.2 

0.18662 

-0,223 

-0.719 

Ok  -0.201 

ov  -0.278 

b: 

+  0.8 

A1.6 

0.78000 

+0.334 
+  0.551 

-0.064 
<^]]  -f  0.167 

-0.449 

Oi;  +  0.001 

at  -0.301 

Oi;.  -0.113 

5.34 

ARMA  (0.2) 

o; 

1.0 

1.0816 

a; :  1.05489 

5.35 

MA  (1) 

n 

0 

0 

4 

m 

1 

1 

0 

[Process  (1)] 

b, 

-0.7 

AI  .7 

-0.70294 

fli  +  0.646 
ai  +0.423 

03  +  0.299 
a,  +0.234 

5.37 

ARMA  (0,1) 

O! 

1.0 

1 .0478 

or;  :  1.041 27 

5.38 

MA  (1) 

n 

m 

0 

1 

°  {  NOT  \ 

1  1  OPTIMUM  ) 

8 

0 

[Process  (2)] 

b, 

+  0.7 

+  0.74320 

fli  -0.71964 

02  +0.49117 

1.0 

1.0442 

oj  -0.30087 
o«  +0.26673 

— 

n 

0 

A1.8 

1 

05  -0.26548 

06  -0.18590 

m 

1 

(OPTIMUM) 

07  -0.09790 

0,  +0.08703 

(<ii) 

0 

+  0.80208 

b, 

+  0.7 

+  1.5161 

(bz) 

0 

+  0.55205 

5.40 

ARMA  (0.1) 

1.0 

1 .0359 

1.03356 

n 

2 

2 

15 

5.41 

ARMA  (2,2) 

m 

2 

2 

10 

a\ 

-0.5 

-  0.40878 

o,  -0.721 
02  +0.192 

09  +0.091 
oio  -  0.252 

ai 

+  0.8 

A1.9 

+  0.80245 

oj  +0.515 
o*  -0.190 

^11  —  0.01 5 
an  -0.158 

b, 

+  0.2 

+  0.25973 

05  -0.456 
06  +0.361 

ot3  +  0.015 
a,t  +  0.098 

bj 

+  0.8 

+  0.95205 

07  +0.177 
o«  -0.237 

an  +  0.105 

5.43 

ARMA  (2.2) 

1.0 

0.84312 

a]  :  1.06480 
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523.4  ARMA(}2)  Process.  The  equation  for  the  ARMA(1.2)  process  is 

X;  +  =  €/+ /?!  €/_i  +  Z?2  ff-2>  (5.195) 

or,  using  the  backward  shift  operator, 

(1  +£Ii5)X,  =  (1  +  M  +  (5.195’) 

Analogous  to  ARMA(2.1),  the  stationaUty  is  determined  by  the  left-hand  side  of  this 
equation,  i.e.,  by  the  autoregression  part.  On  the  other  hand,  the  invertibihty  is  deter¬ 
mined  by  the  character  of  the  right-hand  side  of  Eq.  5.195’.  Accordingly,  the  stationality 
is  the  same  as  for  AR(1)  and  the  inveriibility  is  the  same  as  for  MA(2).  Green’s  function, 
the  inverse  function,  the  autocovariance  function,  and  the  spectrum  function  can  be  ob¬ 
tained  in  the  same  way  as  for  ARMA(2.1). 

5.3  GENERAL  ARMA(n.m)  PROCESS 

53.1  General  ARMAin.m)  Process.  Its  Stationality  and  Invertibihty 
Analogous  to  the  preceding  section,  when 
X, -t- +<22-^/-2  +  •  •  • +^n^/-n  =  f/+^l  f /-I  +  ^2  f/-2+'  '  +b„€,^, 

(5.196) 

the  process  X,  is  referred  to  as  the  nth  order  autoregressive,  mth  order  moving  average 
process,  ARMA(n,ni).  When  the  backward  shift  operator  B  is  used, 

(I  +  aiB  +  a2B^  +  •  •  •  +  (i„  B")X{  — {I +biB  +  b2  B^  +  •  ■  ■•¥bmB"')€t  (5.196’) 


where 


—  1  +  +  *  •  •  +  Qf^ 


(5.196”) 


P{Z)=\+biZ  +  b2Z^  +  -  •  •+  brriZT. 

Green’s  function  is  derived  from 

x-Yg  . 


j  =  0 


a{B) 


and  the  inverse  ftmction  is  derived  as 


e-y-ix  =^X 

j  =  o  '  ' 


(5.197) 


(5.198) 


(5.199) 


By  the  same  logic  as  we  had  for  ARMA(2.1),  for  this  process  to  be  stable,  the  char¬ 
acteristic  function 

/i(Z)  =  Zr  +  aiTT-^  +  +  •  •  •  +  fl„  (5.200) 

equated  to  0,  i.e.,  /i(Z)  =  0  must  have  all  its  roots '  •  -p-n  inside  the  unit  circle, 

or a(Z)  =  1  +  fl|Z  -h  ■  •  •  a„  =  0  must  have  all  its  roots  outside  the  unit  circle. 
Conversely,  for  this  process  to  be  invertible,  the  characteristic  function 
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(5.201) 


/2(Z)  =  b22r-~+-  ■  ■  +  b„.  =  0 

must  have  all  its  roots  vj ,  vo  •  ■  ■  v„  inside  the  unit  circle  or 
^(Z)  =  l+biZ  +  b2Z-  +  -  •  •  +  b^ 
must  have  all  its  roots  outside  the  unit  circle. 

5J.2  Green  's  Function  for  ARMA(n,m) 

Green’s  function  Gj  is  defined  as 

oe 

X,  =  2  Gj  €,-j  =  Go  €,  +  Gi  e,_i  +  •  ■  •  +  Gj  •  •  • 

j  =  Q 

=  (Go  +  GiB  +  G2B~  +  •  •  •  +  Gj  B^  +  ■  ■  ■)€ I . 

Then  Eq.  5.196’  is 

(I  +  OiB  +  02B^  +  •  ■  •  +  arB^KGo^ G\B  +  G2B~  +  ■  ■  ■  +  GjB^+-  ■  •)€/ 

=  (i  +  biB  +  b2B^  +  •  •  ■  +  b„  B'”)e, .  (5.202) 

Equating  the  same  order  of  ^'ackward  shifting  operator  B  for  both  sides  of  Eq.  5.202, 

Go  =  1 

OiGq  +  Gi  =  bi 
oiGq  +  Gidi  +  G2  =  i>2 

flmGo  +  Glflm-l  +  G2am-2  +  ‘  *  '  +  G  - 

anGo  +  Giat_i  +  GjOk-i  •  •  •  +  G*  =  0  for  /:  >  m,  (5.203) 

or 

Go  =  1 
Gi  =  bi-ai 

G2  =  b2-(bi-ai)ai-a2 

G„  =  b„  -  -a„.  (5.204) 

Explicitly  Green’s  function  can  be  obtained  through  the  solution  of  the  homogeneous 
equation 

Xt  +  OlXf-l  +  02^1-2  '  '  'Or  Xi-n  —  0  (5.205) 

with  initial  conditions  gi .  .;n  by  Eq.  5.204,  that  is,  in  the  form 

Gj  =  gi  fi{  +  g2  /i2  +  ■  ■  ■  +  gn  fin  (5.206) 

where  fi-[,  fii'  •  -/tn  are  the  eigenvalues  or  the  roots  of  the  characterization  equation 
/i(Z)  =  Z"  +  aiZ^^  +  022^^  +  •  •  '  a„  =  0. 
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Especially  when  m  =  n-1,  namely  for  ARMA(/i,  n  -  1),  by  Pandit  and  Wu’* 

-  ■  ■  +  b„^i) 

St  ~ 

t  =  1,2,  -  •  n  (5.207) 

where  the  denominator  is  the  product  of  all  terms  in.-fij)  for  i  =  1 . 2,  •  ■  •  ai.  excluding 
the  zero  term  .  Equations  5. 135  and  5.136  are  the  special  cases  j  =  1  and  2  for 

n  =  2.  Each  real  root  fij  in  Eq.  5.206  provides  an  exponentially  dynamic  mode  like  an 
AR(1)  model,  whereas  a  complex  conjugate  pair  of  roots  as  in  Eq.  5.137'  gives  an  expo¬ 
nentially  decaying  sinusoidal  mode,  whose  frequency  and  decay  rate  can  be  obtained  as 
Eqs.  5.140  and  5.1 39  illustrated  in  Fig.  5.24  in  Section  5.2.4. 

53.3  Autocovariance  and  Spectrum  Function  of  ARMA(njn) 

For  the  ARMA(/i,w)  process, 

Xt  = —02X1^2  ~  '  '  OnXi-n  +  €1+  bl  ^>2  €  l-ni  ■ 

Here,  taking  into  account  the  relations 

Xi='^Gj  €,.j  =  Go  (,  +  Gi  €t-i  +  G2  e/_2+-  •  ■  +  Gj€,..j+-  ■  ■ 

^i-r  =  X  €i_^+Gi  +  C?2  C,_^_2  +  •  •  ■  +  Gj  €,_r-j+  ’  '  " 

E[€,-€,.r]=6r  a}  , 

multiplying  both  sides  of  these  upper  two  equations  term  by  term,  respectively,  and  tak¬ 
ing  the  expected  values  gives, 
when  r  =  0, 

/?(0)  =  -ai^(l)-a2W(2) - arJR{n)A{GAbxGiAb2G2^-  ■  ■b„G„)o} 

when  r  =  1, 

/?(!)  = -ai/f(0)-a2^(l)-a3i?(2)--  •  --ajiin-l) 

+  (^?iGo  +  +  &3G2  +  •  •  +  b„Gm-\)o} 

Ri2)  = . 

when  r  =  m, 

R{m)  =  -aiR{m-l)-a2R{m-2)+ ■  •  a„  Rijn-n) 

+  t>m  Go  oi 
and 

R(k)  =  -ayR(k-l)-aiR(k-2)  +  -  ■  ■  +  aji(k-n\  k  >  m+  1.  (5.208) 

Analogous  to  the  case  of  ARMA(2.1),  /JfO)  to  R(k),  k  <  n  -(■  1,  can  be  obtained  by  solving 
these  simultaneous  equations.  Also,  if  necessary,  we  can  get  the  general  expression  for 
R{r),k  >  m  -h  1,  by  generalizing  the  case  of  ARMA(2.1).  By  the  same  procedure  we 
used  for  ARMA(2.1),  the  spectrum  s((o)  for  the  ARMA(n,m)  process  is  naturally 
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obtained  as 


4<u)  = 


ot 


1  +  +  •  •  •  +  b„e- 


2ji 


\ -^  axe-^  +  a2e-^  +  ■  ■  -^a. 


,-n  t  w 


(5.209) 


(5.209’) 


53.4  Estimation  ofaj  .  . .  Cn,  bj  .  . .  b„  of  ARMA(n,m) 

For  starting  values  of  ai  ■  ■  -On,  bt  ■  ■  ■  b„,  we  can  get  a  first  approximation  of  the 
estimates  from  Eq.  5.208.  Namely,  with  the  use  of  n  simultaneous  equations  on 

Jt(m  +1),  /t(m  +  2)  •  ■  ■  /i(m  +  n)  for  ^  m  +  1 

autoregressive  parameters  a\,  02,-  ■  ■  a„  can  be  estimated,  although  the  less  reliable  val¬ 
ues  of  autocorrelation  at  larger  m  +  n  must  be  used.  Then,  inserting  these  values  in  the 
first  (ffi  +  1)  equations  inEq.  5.208  onl?(0),/?(l)  •  •  Rim),  m  +  1  unknown  parameters 
hi  •  •  •  bjn  and  ai  can  be  estimated  theoretically  by  solving  the  nonlinear  equations  on 
these  parameters. 

The  necessity  for  solving  nonlinear  equations  can  be  shown  as  follows:  In  the 
expression  for  ARMA(n,m) 

Xi  +  OiXt-x  +  02X^2  +  ■  ■  •  +  a,Xi-n  =  +  b\€t^x  -(-•••  -I- 

using  recursively 

€f-i  =  X/-1  +  aiXt-2  +  02^1-3  +  •  •  •  +  a,^iXt^  + 

-bl€,.2-b2€,-3--  •  -bm€i-m^i 

in 

Xi  =  —  a  lX(_i  —  02^1-2  ■  ’  ~OrXt-n 

+  bi€t-i  +  b2€i^2'  ■  • + 

Xt  =  {-ai  +  bi)X,.i  +  i-a2-axbi)X,^2  +  ’  '  • + 

+  ibi-b2)€t-2+ibxb2-b3)€,.3+-  '  •  +  bibme,^m^i  +  e,.  (5.210) 

We  still  have  the  terms  e,_2  ,e^3  •  -  -  that  should  be  expressed  by  X^2.  X^3  •  ■  •  and  also 
f  (-2 .  ^1-3  •  When  the  dependence  of  X  is  expressed  in  terms  of  past  X>  the  equation  will 

be  nonlinear  in  the  unknown  parameters  u  1, 02*  •  •  andh],  h2<  •  •  •.  because  their  prod¬ 
ucts  and  squares  are  involved.  Thus,  the  regression  becomes  nonlinear  and  requires  a 
nonlinear  least  squares  method  for  estimation. 
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Actually,  however,  it  is  troublesome  to  solve  the  nonlinear  equations,  and  other  ap¬ 
proximation  methods  are  used.  An  approximation  method  introduced  by  Pandit  and  Wu^ 
is  to  use  the  inverse  function,  taking  advantage  of  the  linearity  between  the  unknown 
coefficient  and  the  inverse  function  as  shown  in  Eq.  5.150  for  ARMA(2.1).  In  this  meth¬ 
od,  we  first  fit  the  pure  autoregressive  model  of  an  appropriate  large  order  AR(A)  to  this 
ARMA(«,/n)  process  and  then  use  the  coefficients  of  this  AiR(k)  model  as  an  approxima¬ 
tion  for  inverse  functions  at  larger  values  of  k.  Then  from  the  equation  that  connects  bj 
and  Ij,  we  can  gradually  improve  the  approximation  of  bj  and  Oj  and  fmally  adjust  the 
results  from  the  viewpoint  of  invertibihty.  The  procedure  is  rather  comphcated,  and  care 
is  necessary  to  get  good  estimates.  In  any  case  estimation  of  the  coefficients  for  ARMA 
(n,m)  is  not  necessarily  easy. 

Estimation  of  the  parameters  for  the  pure  autoregressive  process  AR(n)  is  more 
straightforward,  as  we  saw  for  AR(2)  in  Section  5.2.3  or  as  will  be  shown  in  Section  5.4, 
since  it  can  be  done  through  linear  regression. 

In  practical  applications  of  the  model  fitting  technique,  the  pure  autoregressive  pro¬ 
cess  AR(k)  is  frequently  used  as  the  model  to  be  fitted,  although  the  order  k  sometimes 
becomes  a  little  l^ger  than  n  of  ARMA(n.m),  which  should  be  the  acnial  model  to  be 
fitted  to  the  process  imder  consideration.  We  saw  this  tendency  in  examples  of 
ARMA(2.1),  MA(2),  MA(1),  and  ARMA(2. 2)  shown  in  Figs.  5.27,  5.30,  5.34,  5.37, 
5.40,  and  5.43. 

SJJ  Adoption  ofARMA(n,  n- 1),  ARMA(2n,  2n  -1 ) 

As  was  mentioned  in  Sections  5.2.2  and  5.2.4,  Green’s  functions  and  the  autocovari¬ 
ance  functions  are  in  the  form  for  AR(1);  Gj  =  -  fi'\  R(r)  =  and  for  ARMA(2.1  ): 

S\  +  ^2  R(f)  =  Bp{  +  B2  pi'  3nd  show  increasing  complexity  as  the  order 
increases.  For  AR(2),  as  derived  in  Section  5.2.3,  Gj  and/?(r)  were  of  the  same  type  as 
those  for  ARMA(2.1).  However,  as  was  mentioned  in  Section  5.2.4,  ARMA(2.1)  is  rec¬ 
ommended  by  Pandit  and  Wu^^  as  a  more  general  process  that  is  made  up  of  two  expo¬ 
nentials.  The  other  reason  for  preferring  ARMA(2.1)  over  AR(2)  is  the  fact  that  the 
ARMA(2.1)  process  is  more  closely  related  to  the  system  that  is  governed  by  the  second 
order  differential  equation,  as  will  be  explained  in  Section  6.3.2.  In  the  same  way,  for  a 
more  complex  process  made  up  of  three  exponentials,  AR(3.2)  is  the  most  general  pro¬ 
cess.  Extending  this  relation,  ARMA(/j,  n-l)  is  recommended  by  Pandit  and  Wu^ as  the 
most  general  process  when  the  process  is  represented  by  nth  order  dynamics  as 

Gj  =  81  Mi  +  gi  fi2  +  -  ■  ■  +  8nfiJ-'  Rir)  =  Bi  pj+B2  p2+-  ' 

The  reason  that  n-l  should  be  the  order  of  the  moving  average  part  is  also  shown  in 
Pandit  and  Wu.^  Furthermore,  as  they  pointed  out,  empirical  experience  indicates  that  it 
is  better  to  increase  n  in  steps  of  two  and  fit  ARMA(2n,  2n  -  1 )  models  for  n  =  1 , 2,  3. 

Also,  if  a  complex  root  appears  among  pj,  and  <21,02  •  •  •  are  to  be  real,  complex  roots 
must  appear  as  conjugate  pairs  as  was  shown  for  ARMA(2.1)  in  Section  5.2.4.  Therefore 
to  fit  an  ARMA(2n,  2n  -  1)  model  is  more  practical  than  to  fit  an  odd  order  model. 
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5.4  GENERAL  AUTOREGRESSIVE  PROCESS  AR(n^ 

5.4.1  Adoption  ofAR(n)  Model 

The  general  autoregressive  model 

X,  +  a\Xt-\  +  a2^»-2  +  •  •  ■  +  =  (5.21 1) 

is  considered  to  be  a  special  case  of  the  general  autoregressive  moving  average  model 
ARMA(«,m)  when  m  =  0.  If  invertibility  is  satisfied  for  the  ARMA(n.w)  model. 


=  X,-hX,.,-h  X,_2 - Ij  X, - 

This  expression  means  that  ARMA(R,m)  is  transformed  into  an  autoregressive  model  of 
infinite  order  AR( « ).  Practically,  if  a  large  enough  n  is  taken,  almost  all  of  the  ARMA 
(«,m)  process  can  be  approximated  by  an  AR(n ')  process  if  we  assume  invertibihty. 

The  general  character  of  the  AR(«)  process  will  be  summarized  here,  as  AR(/i)  is 
the  most  common  model  used  in  the  practical  application  of  model  fitting  techniques  as 
was  already  mentioned  in  Section  5.3.4.  The  estimation  of  parameters  is  much  easier  for 
the  AR(n)  process  than  for  the  MA(m)  or  AR(«,m)  processes. 

5.42  Green’s  Funaion  of  AR(n) 

The  AR(n)  process  is  expressed  as 

Xt  +  OiX,^i  +  02X1^2  +  ■  ■  ■  +  fln  =  Cf  (5.21 1 ) 

a+aiB  +  a2B^  +  -  •  ■  +  a^")X,  =  €,  (5.211’) 


a(B}X,  =  €,. 


(5.211’) 

(5.211”) 


a(Z)  =  1  +  aiZ  +  02^  +  •  •  Un  27'. 
Using  Green’s  function  Gj  gives 


=  {Go  +  GiB  +  G2B^  +  •  •  •  +  GjB^  + 


(5.212) 


X,  =  ^  Gj  €,^j  =1  ^Gj  &  |e,  -  Go  +  Gi  e;_i  +  G26,_2+  •  •  •  +  Gy  e,_y 


(5.213) 


{\  4- QiB  +  Q2B'^  A  •  •  •  +  Qfji’') 


(5.213’) 


Equating  the  same  power  of  B  on  both  sides. 
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Go=l 

GgOl  -f  Gl  =  0 

^iGo  +  ^iGi  +  G2  =  0 
^^Go  +  02G1  +  aiG2  +  G3  =  0 

flnGo  +  <ln-lG\  +  •  •  •  +  G„  =  0, 


(5.214) 


or 


Gi  = 

G2  = 

^3  = 


-ai 

-a2  +  ai 

-  03  +  0201  +  01(02  -  of)  =  -  03  +  20i02  -  of. 


(5.215) 


Generally 

(o„B"  +  •  •  •  +  1)G„  =  0. 
Explicitly  as  shown  in  Section  5.3.2 

Gj  =  gl  M{  +  g2f^2  +  -  •  •  +  gnMn 


(5.216) 


where yui  •  •  /tn  are  the  eigenvalues  or  the  roots  of  the  characteristic  equation  equated  to 
zero  as 


/i(2)  =  2"  +  oi2^^  +  •  •  •  +  a„  =  0 


and 


gi  = 


(5.217) 


(5.218) 


~ f^2)  '  ■  ’  i  Atj+1)  ’  ■ 

where  the  denominator  is  the  product  of  all  terms  fii  -  •  -fij  for  j=\,2  -  •  n,  excluding 

the  zero  term;*, as  the  special  case  where &i  =  62  =  ^3  =  •  •  •=  hm  =  OinEq. 
5.207. 

5.43  Autocorrelation  and  Spectrum  Function  for  AR{n) 

For  AR(«) 


Therefore 


Xt - —  <17X1-2  •  •  •  —  0;,X,_„  +  Cf  . 


Xf-r - 0\Xi~r-\~07Xt-T-2~  '  '  •  —  +  f  j-r  • 


(5.219) 


Multiplying  both  sides  of  these  two  equations,  respectively,  taking  the  expected  values 
gives 
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/?(0)  =  - <32^(2) •  --an  Rin)  +  oi 

R{l)  =  -aiR{0)-a2Ra)-  • 


(5.220) 


Rik)  =  -aiR(k-\)-a2Rik-2)-  ■  -ajiik-n) 


k  =  1  ■  ■  ■  n. 

Using  the  backward  shift  operator  B  gives 

B>R{k)=^R{k-j). 

Then  the  last  equation  in  Eq.  5.220  is 


R{k)  = 


\R{k) 


(5.221) 


(5.222) 


or 

n 

&)R{k)  =  0.  (5.222’) 

7  =  0 

Equation  5.222  is  very  similar  to  the  expression  of  the  homogeneous  equation  of  the 
process  itself  in  Eqs.  5.211  5.211”  when  e,  =  0  as 


a{B)R{]c)  =  0.  (5.222”) 

Therefore,  as  was  Eq.  5.166  for  AR(2),  the  general  form  of  R(ifc)  is 

R{k)  =  5i  +  B2  +  •  •  •  +  ^"  (5.223) 

Here  are  the  eigenvalues,  and  •  •  •  B^  are  constants  that  can  be 

obtained  theoretically  from  the  boundary  conditions 

f?(0)  =  +  ^2  +  •  •  •  +  B„ 

R(l)=Bi  fii+Bi  fl2+'  • 

(5.224) 


The  spectrum  is,  as  given  by  Eq.  5.209, 
s(<o)  = 


o} 


2jt 


a(e-^) 


2jt 


l+aie-^  +  a2e-^  + 


+  a„e 


-two 


(5.225) 
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5.4.4  Estimation  of  Parameter  aj  . .  .a„  and  ft 

In  this  section,  we  assume  that  the  order  n  of  the  autoregressive  process  AR(/i)  is 
already  known.  Determination  of  the  order  n  is  discussed  in  the  next  section.  Here  the 
statistical  considerations  for  the  estimation  are  treated  in  some  detail,  compared  with  the 
case  for  AR(2)  in  Section  5.2.3.  As  the  most  general  type  of  AR(«)  process,  we  assume 
the  nonzero  mean  process  to  be 

(Xi -ju)  +  ni(X,_i -ju)  +  a2(X,.2 -ft)  +  •  •  •  +  CniX^n  -fi)  =  €[.  (5.226) 

By  analogy  with  the  multivariate  regression  of  X,  with  X-\-  ■  •  X,  with  regressive 
coefficients  ai,  02-  •  ■  a„  and  residual  error  e, ,  we  can  use  the  least  squares  method  to 
estimate  ai  -  ■  ■  Oo,  ft,  ande, ,  that  is,  to  minimize 

N 

■  -  On)  =  X 

t  =  n+l 


=  y  {(^/  -  Ai)  +  ai{X,.i  -fi)  +  •  •  •  +  a„(X,^  -Ai)p.  (5.227) 

I  =  n+l 


The  reason  for  this  lower  limit  n  +  1  is  that  we  do  not  have  observations  Xo, 

•X-i,  ■  •  •  X_  _  j)  from  which  to  derive  €<> ,  •  •  ■  N  is  the  total  number  of  observations 

of  X).  The  residual  error  €<  is  assumed  to  be  Gaussian  and  independent  of  each  other. 
Accordingly,  the  joint  probabUity  distribution  of  6„+i  •  •  is 


/7(€„+i,€„+2-  *  -^n)  = 


n 


e 


ff 


r  =  n+l 


(5.228) 


(5.229) 


Here  the  variables  are  transformed  from  (e„+i,e„+2-  •  •€„)  to  (X„+i,Z„+2. '  '  ’^Af) 
using 
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€n^\  =  {Xn^i-fx)-¥ai{X„-n)-¥a2iX„_i-n)-  ■  ■  +  a„{Xi-n) 
f  n+2  =  (■X^n+2  “/W)  +  ai(X„+i -^)  +  a2(^n -/W)  +  •  '  ■+a„(X2-fi) 


(5.230) 


[es  =  iXs-fj.)  +  ai(XN.i-fi)  +  a2(XN^2-f^)+ -  '  ■  +  a„(Xs-n-f^)- 


FromEq.  5.230  the  Jacobian  to  transform /j(e„^  ,,e„  ^2  ■  ‘  p(X„^i,X„+2-  '  -Xfj) 

is 


d(€„+i€n+2'  •  •  fAf) 

diX„^iX„^2-  ■  -Xn) 


Therefore  from  Eq.  5.229, 
p(Xf,^i,Xf,^2> '  '  '  Xfi/)  = 


(5.231) 


exp- 


j]  j(X,-Af)  +  ai(X,_i-/i)  +  -  •  •  +  an(Xt-n-p)f 

/  =  n+l 


01,02-  •  -  On) 


(5.232) 


This  is  actually  the  conditional  joint  distribution,  given  that  the  initial  observations 
Xi  ■  ■  -Xn  remain  fixed  at  their  realization  X]  ■  ■  ■  X„.  Here  n  is  assumed  to  be 
relatively  small  compared  with  the  number  of  observations  N.  We  can  then  use  this 
p(X  „4.i ,  •  •  •  X  TV )  as  the  adequate  approximation  of  likelihood  function  of 
fi\,a\,a2-  ■  •  fln,  given  the  observations  Xj,  X2  •  •  •  A^.  Then  the  log-likelihood 
function  L  is 


•  ■a„)  =  logp(X„+i,X„+2,-  *  -Xn) 

N 

=  ”hog(2;r  a})  S  {{X,-|4)  +  ai(X,_i  -p)  +  -  •  •  +  a„{X,^  -/^))^ 

2  2ae  r  =  n+ll  > 

(5.233) 

Therefore  the  maximum  likelihood  estimate  for  p\a\  •  •  -  is  to  maximize  L,  i.e.,  to 

minimize 

N  2 

•  ■  •<2„)=  ^  |(X,-^) +  ai(X|:_i-;f)  +  •  •  • +  a„(X,_„-^)|  .  (5.234) 

/  =  n+l 
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This  is  the  same  as  adopting  the  least  squares  estimate  as  already  mentioned.  That  is 
obtained  from 

dQ 


and 


dM 


dQ 


=  0 


^  =  0  j  =  l,-  n 
da. 


as 


N 

2 

t  =  n+l 


^  f  1 

J  +  +  -  ■  ■^a„{X,_„-n)\  =  0  (5.235) 


and 

^  r  1 

^  |(X, -n)  +  ai(X,_i  -/r)  +  •  •  •  +  afX,-j-fi)  •  •  ■  +  a„(X,.n  -fi)  )(X,_;  -/r)  =  0. 

(5.236) 

From  Eq.  5.235 

Xo  + 61X1+02X2-  •  •  +  a,^„ 


fi  = 


1+61+62  +  -  •  ■  +  d„ 


(5.237) 


where 


1  ^ 

/  =  «+! 


_  1 

When  n  is  small  compared  to  A^i,  Xj  will  be  close  to  X  =  —  2^Xt  and  Eq.  5.237  will  be 


N 


£-1 


fi=X. 


(5.238) 


From  Eq.  5.236 
N 


2  {(X,-X)  +  ai(XM-X)  +  -  •  -+6jiX,.j-X)-  -  -  +  6n{X,.„~X)]  X 
{(X,_,-^)  =  0,  j=h-  -  -n. 


/  =  n+l 


Here,  if  we  approximate  further,  under  the  assumption  that  k  is  also  small  in  relation  to  N, 
then 

N 

X  (.X,^-iO(.X,.j-X)  =  NR(j-k), 

t  =  n+l 

and  Eq.  5.236  becomes 
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R(j)  +  aiRiJ-l)  +  -  ■  ajRiO)-  ■  ■ -i- a„R(J - n)  =  0  (5.239) 

j=l  -  ■  n. 

From  this  relation  <21,^2,-  •  -dj-  •  •  can  be  estimated  by  solving  the  simultaneous 
equations 

tJl/?(0)  +  +  ■  ■  ■  —  1)  =  — /?(1) 

<21^(1 )  + ^2^(0)  +  •  •  ■ +a,R(n-2)  = -R(2) 

*  '  (5.240) 

l)  +  fl2^(«-2)  + •  ■  ■  +  a,fi{0)  =  -R{n). 

This  can  be  transformed,  using  the  vector  expression,  to 

(ai,a2,-  •  -an)' =  a  (5.241) 


|i(l)^(2)-  •  -.R(n)j'  =  r 

'  RiO)  Ril)--  Mn-1) 
R{1)  '  '  R{0)-  •  •/?(«- 2) 

A  ' 

.;R(/2-1)  Rin-2)  R(0)  . 
A 

Rn  a  =  -r. 

Therefore 

.  —r 
a  = - . 

A 

Rn 


(5.242) 


(5.243) 


(5.244) 


(5.245) 


Equation  5.239  or  5.240  is  identical  with  the  Yule-Walker  equations. 

All  these  relations  mean  that,  if  the  approximations  as  given  here  are  introduced,  the 
maximum  likelihood  method  can  be  replaced  by  the  least  squares  method,  and  the  least 
squares  method  is  reduced  to  get  the  solution  of  the  Yule-Walker  equations.  Priestley^ 
describes  in  more  detail  the  precise  estimation  through  a  rigorous  likelihood  function. 


192 


5.45  Estimation  of  o}  for  AR(n) 


A' 

a}  is  estimated  as  the  variance  of  the  residual  error  e,  from  Q  =  ^  f/"- 

Because  {N  -  n)  observations  are  summed  and  n  +  1  parameters  (ji  and  naj ’s)  have  been 
estimated,  the  unbiased  estimation  is 


1 

iN-n)-{n  +  1) 


QiAJiJz,-  •  an) 


1 

N-2/J  +  1 


X  [(X,  -X)  +  -X)  +  ■  ■  ■  +  d„{X,^  - D j' 

/  =  n+l 


=  —^^—^\R(0)  +  aiRa)+-  •  •  +  a„  /?(«)]  .  (5.246) 

-  2n  +  1  L  J 

When  n  «  N, 

ai  =  R(0)  +  aiRil)  +  ■  ■  ■  +  ajiin)  (5.246’) 

This  result  is  the  same  as  we  obtained  from  the  first  equation  of  Eq.  5.220. 

5.5  DETERMINATION  OF  THE  ORDER  OF  THE  FITTED  MODEL 

As  was  explained  in  the  preceding  sections,  the  process  becomes  more  complicated 
as  the  order  increases.  Thus  AR(2)  is  more  complicated  than  AR(1)  and  ARMA(3.2)  is 
more  complicated  than  ARMA(2.1).  Estimation  of  parameters  has  been  discussed  in 
each  section,  so  here  only  the  determination  of  the  order  will  be  discussed.  Several 
different  methods  have  been  described  by  Priestley.^^  The  MAIC  method  developed  by 
H.  Akaike  is  now  considered  to  be  the  most  reasonable  from  the  point  of  view  of  statisti¬ 
cal  considerations  and  of  information  theory  and  is  explained  and  reconunended  in  the 
following  sectiotL 

5JJ  Residual  Error  Method 

The  change  of  residual  error,  expressed  for  example  by  Eq.  5.246’  for  AR(rt),  is 
investigated.  If  the  order  is  far  smaller  than  the  true  order,  <7^,  the  residual  error,  will 
decrease  considerably  as  the  order  increases  from  j  ton.  From  there  on,  the  decrease  in 
residual  error  may  not  be  significant  but  will  remain  at  the  same  level.  Point  n  is  adopted 
as  the  proper  order  of  the  model. 
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Rg.  5.44.  Residual  error. 

Sometimes,  however,  it  is  not  easy  to  determine  the  order  because  the  rate  of  change 
does  not  show  clearly  by  this  method. 

5-52  Partial  Autocorrelation  Method 

If  denotes  the  i  th  autoregressive  parameter  of  an  AR(«)  process,  then  the  auto¬ 
correlation  coefficient  p,  is  from  Eq.  5.220, 


Qi  ~  ^nlQi—1  ^rOQi-l  '  '  '  ^nnQi-n 
I  =  1,2,-  •  n 

which  gives  the  Yule-Walker  equations. 


(5.247) 


^nlQl  '  ^nnQn-1 

Q2~  ~^nlQl~^n2Qo'  ■  '~OnnQn-2 
\  (5.248) 

jQn  ~  ~  O/fiQn—l  ~  On^n-2  '  '  '~^nnQo- 


Then,  the  last  coefficient  Om  of  the  fitted  AR(n)  model  is  called  the  partial  autocorrelation 
p„',  for  n  =  1,  2  •  •  • ,  and  will  be  used^’  to  suggest  the  order  of  the  AR  model. 

Since  pt,  =  1,  from  Eq.  5.248, 

-aii=Pi-*Pi' 


1  Qi 
Qi  Qi 

1  Qi 
Q\  1 


Pi-g2 

1-q\ 


Qi 


(5.249) 
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1  Ql  Q2 
Qi  1  Pi 


■  P»-2  Pi 
•  Pn-3  P2 


1  P»i-1 

P»-l  Pn-2  Pn-3  '  '  •  Qn 

1  pi  Q2  ■  ■  Pn-2  Pn-1 
Pi  1  P.  •  •  •  Pn-3  Pn-2 


—  Pn 


(5.249 


I  Pn-l  Pn-2  Pn-3  ‘  '  Pi  1  | 

A  plot  of  p  1  'p2' '  ‘  '  Pn'  against  n  is  called  the  partial  autocorrelation  function  diagram. 

If  the  true  order  was  n,  p*'  or  -  a*  *  should  vanish  for  it  >  n  and  the  plot  of  the  partial 
autocorrelation  should  show  the  real  order  n. 

For  this  purpose,  a  recursive  method  for  calculating  p„'  when  -  a„„  are  estimated 
from  Yule-Walker  equations  has  been  given  by  Durbin^®  as 

^n+l»/ ~  ^nj~^n+l>n+l^n»n-/t-l  »  (5.25 

n 

Q(m  +  D-  5]  a„  jQ{n  +  1  -;) 


1  -  S  «n.y  PO) 
j=l 

5SJ  Visual  Inspection  (rf  the  Autocorrelation 

As  has  been  shown  in  the  preceding  sections,  the  types  of  models  and  their  orders 
are  well  reflected  in  the  pattern  of  the  curve  of  the  autocorrelation  function.  For  example, 
for  the  MA(m)  model,  the  correlation  cut  down  to  0  after  the  lag  m,  and  for  the  AR(m) 
model,  the  correlation  has  the  form  of  damping  oscillations.  Box  and  Jenkins^^  have 
developed  this  method  and  showed  many  examples  of  estimating  the  order  from  these 
patterns.  However,  it  requires  skill  and  long  experience  to  be  able  to  estimate  the  actual 
order  correctly  by  this  inspection. 


55.4  Akaike’s  FPE,  AIC,  and  BIC  Criteria 

FPE.  Akaike^  first  proposed  computing  the  final  prediction  enor  (FPE)  in  deter¬ 
mining  the  order  n  of  AR(«)  as 

FPE(^)  =  ^-6}  (5.252) 

N-q 

where  N  =  number  of  observations  to  which  the  model  is  fitted.  q  =  the  order  of  the  AR 
model 

=  1^(0) +  1^(1) -t- •  •  ■+a^{q) 

and  adopting  as  n  the  value  of  q  that  minimize  the  FPE,  as  in  Fig.  5.45. 

FPE  (q) 

t 


Fig.  5.45.  FPE  vs.  q. 

AIC.  Later  Akaike^'"*^  proposed  a  more  refined  method  of  minimum  AIC 
(Akaike’s  Information  Criteria)  based  on  information  entropy  theory.  This  is  a  ver>'  gen¬ 
eral  concept  available  for  general  statistics  problems,  as  an  example  of  curve  fitting,  that 
is  shown  in  Appendix  A-2.^*  AIC  is  defined  as 

AIC(^)  =  (-  2)loge[Max.  Likelihood]  +  2q  (5.253) 


where  ^  is  the  number  of  parameters.  Including  dc ,  n  -t- 1  parameters  have  to  be 
estimated,  so 

q  =  nA\.  (5.254) 

The  loge  [maximized  likelihood]  is  L  in  Eq.  5.233  or,  omitting  the  constant, 

-  N-n  2  1  ^ 

Here  Q  is  defined  as  the  sum  of  squares  and  from  Eq.  5.246 


Thus 


L  = 


Ignoring  the  second  term,  which  is  constant,  and  inserting  it  in  Eq.  5.253 


(5.255) 
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AIC(<7)  =  (-  2)^-—^  ^  loggf  j  +  2^ 

=  (A'-«)loga/+ 2(/2  +  1).  (5.256) 


Rg.  5.46.  AlC(q)  vs.  q. 

The  AIC  (q)  is  plotted  against  /i  as  in  Fig.  5.46,  which  shows  where  the  AIC  (n)  has 
a  minimum  value.  Tlus  minimum  value  is  called  MAIC,  the  minimum  AIC,  and  this 
method  for  obtaining  that  MAIC  is  called  the  MAIC  method.  Akaike^*’^* made  clear  that, 
based  on  information  theory,  AIC  is  a  measure  of  close  fimess  of  approximation  =  a 
measure  of  minimum  difference;  (statistical  model-true  model). 

So  if  we  express  the  statistical  distribution  of  the  occurrence  of  the  data  for  the 
model  as  p(x)  and  the  statistical  distribution  of  the  true  data  derived  from  the  true  struc¬ 
ture  of  the  data,  as  q(x),  to  minimi?/!  AIC  is  to  minimize 

7=  [p(x)log^, 

J  gix) 

the  so-called  Kulback’s^^  information  criterion.  Also  the  so-called  information  entropy 
S(q,p)  is  defined  as  S(q,p)  »  -/.  So  to  minimize  AIC  is  to  maximize  the  entropy 
Siq,p).  Accordingly,  the  minimum  AIC  criteria  are  also  called  the  maximum  entropy 
criteria. 

FPE  was  introduced  to  find  the  order  of  the  AR  model.  AIC  is,  however,  a  very 
general  criterion  and  can  be  used  for  ARMA  or  MA(ot)  models  too.  For  the  ARMA 
(n,m)  model,  we  can  use  this  criterion  setting 

AIC(/i,m)  =  {N  -  m  -  n)logai+  2{n  +  m  +  !)•  (5.257) 

Plotting  AIC(«,m)  over  an  appropriate  grid  of  n,  m  we  can  adopt  (/i,m)  that  minimi?/ 
AIC(/i,m). 
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Akaike  also  made  it  clear  that,  for  large  N,  FPE.  and  AIC  are  equivalent,  because 


logFPE(^)  =  log  <7; 

=  log(  1  +  q/N)  -  log(  1  -  q/N)  +  log  6; 

=  log<7(  +  2.9/A^  (for  large  A^).  (5.258) 


Therefore  AIC(<7)  =  iVlog[FPE(^)]. 

When  N,  the  number  of  observations,  is  large  enough,  the  FPE  criterion  is  the  same  as  the 
AlC(q')  criterion. 

BlC(q).  Akaike^  "*’ also  proposed  the  BIC(^)  criterion,  based  on  the  Baysian  con¬ 
cept,  as  a  new  criterion  for  determining  the  order.  That  is, 

BIC(^)  =  iVlog  a}-(N -q'llogfl  -  q/N)  ■•rqlogN 


r'2  ' 

+  <7log' ►. 

I  l<^€  J 

This  equation  can  be  modified  as  follows  if  we  approximate 

((-iV  -  ^)  logd  -  q/N)]  =  {-  (N  -  q)i.-  q/N)]  =  q-  q^/N  =  q 


(5.259) 


for  N  »  q.  Then 


BIC(9)  =  AIC(^ )  +  ^(log  N  - 1 )  -b  9  log 


=  [iVlogffe-t-  2^]  -i-^(logiV)-9-»-^log 


(5.260) 


This  relation  shows  that  the  difference  between  BIC(9)  and  AIC(^)  is  approximately 
one  q  of  AIC(^)  and  is  replaced  with  q  log  N.  This  replacement  has  the  effect  of  increas¬ 
ing  the  weight  attached  to  the  penalty  term  which  takes  account  of  the  number  of 
parameters  in  the  model.  Shibata^  tells  us  that  AIC(^)  slightly  overestimates  and  BIC((7) 
slightly  underestimates  the  real  q  value. 

5SJ  Examples  of  Order  Determination  through  MAIC 

In  Sections  5.2.1  to  5.2.5,  examples  of  the  synthesized  processes  AR(0),  AR(1), 
AR(2),  ARMA(2.1),  MA(2),  MA(1),  and  ARMA(2.2)  generated  by  mathematical  models 
were  shown.  When  these  processes  were  given  as  observed  data,  the  orders  N  and  M  of 

each  process  were  estimated  by  the  MAIC  method  and  then  parameters  a\-  ■  •  a„; 
b\-  •  ■  bm  were  estimated  by  the  method  described  at  each  subsection.  The  results  are 
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summarized  in  Tables  5.1  and  5.2.  From  these  results,  we  can  find  the  criteria  of  MAIC 
that  give  us  the  correct  values  of  the  order  except  in  a  few  cases.  Estimation  of  parame¬ 
ters  a\  •  •  ■  On,  b\  ■  ■  ■  bn,  andcTf  is  rather  reasonable  in  these  examples.  Tables  5.1  and 
5.2  list  the  AR(n)  model  fitted  to  the  original  ARMA  or  MA  models.  It  is  interesting  that 
the  values  of  N  obtained  by  the  MAIC  method  are  somewhat  larger  than  n  or  m  of  the 
original  processes.  These  models  are  the  AR  models  which  approximate  most  closely  the 
original  ARMA  or  MA  models.  We  can  say  these  examples  gives  us  good  proof  that  the 
model  fitting  method,  supported  by  order  determination  through  AIC  criteria,  is  a  very 
reasonable  and  powerful  method  for  analyzing  the  linear  stochastic  processes. 
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CHAPTER  6 


MODEL  FITTING  TO  THE  RESPONSE  OF  THE  LINEAR  DYNAMIC 
SYSTEM  TO  IRREGULAR  INPUT 


6.1  INTRODUCTION 

In  Chapter  5,  the  characteristics,  stability,  and  invertibility  of  a  single  linear  stochas¬ 
tic  process  X,  were  discussed  and  it  was  concluded  that  AR,  MA,  or  ARMA  models  of 
appropriate  order  can  usually  be  fitted  to  most  of  the  linear  stationary  time  series  to 
represent  their  statistical  properties. 

The  characteristics  of  the  model  fitting  techniques  is,  the  author  believes,  that  all  of 
these  AR,  MA,  and  ARMA  models  relate  the  process  X,  to  the  pure  random  process 

€t ,  although  the  relations  that  connect  €,  to  X,  are  different  for  the  different  models. 

For 


AR  :  Xt  +  CiXi^i  +  aiXt^i  +  .  .  .  +  ci,Xi-n  =  f  / : 

MA:  X,  =  €t  +  +  ^>2€f-2  •  •  •  and  (6.1) 

ARMA:  Xt  +  OiXt~i  +  02X1^2  •  •  •  "^^nXi-n  — 

+  b2€i-2  +  ...  +  bm^i-m  ■ 

In  all  of  these  models,  e,  was  supposed  to  be  the  input  to  an  imaginary  system,  and 
X  was  treated  as  the  output  of  the  same  system.  All  the  characteristics  of  X  are  expressed 
in  relation  to  those  of  the  pure  random  (white)  process. 

6.2  RESPONSE  SYSTEM  WITH  FEEDBACK 

62.J  One  Input /  One  Output  System 

Here  we  consider  the  ouq)ut  T  of  a  real  linear  system  with  input  X,  usually  ex¬ 
pressed  as 


Y,=  ^  guX,.u  + el-  (6-2) 

w=0 

Here  gu  is  the  so-called  impulse  response  function  of  T,  to  X,  or  the  weighting  function  of 
this  linear  system  to  X.  Now  suppose  we  have  N  observations  of  the  input/output  pair 

r=l  to  N. 

If  e/  is  a  pure  random  (white)  noise,  then  we  can  estimate  the  coefficient  g„  to 
minimize. 
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(6.3) 


S 


y, 


n=l 


i<=0 


However,  €,'  is  not  necessarily  white,  and  moreover  it  may  sometimes  have  some 
feedback  effect  on  the  input 

As  already  discussed  in  Section  2.6.3,  even  when  noise  exists  in  the  output  Y,  of 
input  X,,  if  there  is  no  feedback  effect  as  in  Fig.  6.1,  we  can  get  rid  of  the  effect  of  noise 
in  computing  the  real  spectrum  of  the  output  by  finding  the  real  response  function  of  this 
system  from  the  cross  spectrum  of  the  output  to  the  input. 


LINEAR  SYSTEM 


m!>1- 


INPUT 


OUTPUT 


Y, 

OBSERVED 


Rg.  6.1.  Linear  system  without  feedback. 

However,  if  somehow  this  output  is  fed  back  to  the  input,  as  in  Fig.  6.2,  the  input  A', 
is  no  longer  uncorrelated  with  Nt,  and  Nt  is  included  in  Ai  as  a  part  of  the  input.  Accord¬ 
ingly,  taking  the  cross  correlation  does  not  help  in  getting  the  real  response  characteristic 
of  this  system. 


O 


INPUT 


LINEAR  SYSTEM 


N, 


Y, 


OBSERVED 


Flt^  0.2.  Linear  system  with  feedback. 

By  a  parametric  method  different  fi’om  the  nonparametric  method  referred  to  above, 
we  can  obtain  the  real  relation  by  fitting  an  appropriate  model  to  this  system,  getting  rid 
of  the  effect  of  noise  by  following  Akaike.^* 

Under  these  circumstances,  we  first  transform  the  noise  e/  into  pure  white  noise. 
Generally  e/  can  be  expressed  by  an  AR  model  of  arbitrary  order  /, 
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e/  +  Cie,li  +  C2€iL2  .  -  .  +  C/e,l/  =e,, 


(6.4) 


€,  being  a  pure  random  process. 

From  Eq.  6.2 

00 

=Yr-J^guX,^,  (6.2’) 

00 

e,L/  =Y,.i-J^g,X,^^.  (6.5) 

x#=0 

Inserting  these  values  into  Eq.  6.4  gives,  where  /  -*  oe 


^  ' 

00 

00 

'  » 

00 

-  +  Cl" 

Yt~l  “  ^ 

•  +C2' 

^/— 2  “  ^  Su  -^/-2-m 

V  J 

M=0 

^  J 

M=0 

L  ^ 

+  C, 


n=0 


^  +  .  .  .  =  e,, 


(6.6) 


or 


y,+ Ciy,_i  +  C2i^f_2  +  . . .  +  C/y,_/  +  . . . 

00 

-  Xt-i^  +  C2  gu  X,.2-u  +  •  • 

+  C/  gu  x,.^  +  .  .  .  I  =  €,, 


tt=0 


(6.7) 


or 


^ Cl  Yt-i  =  ^gu^Ci  Xt-u-i  +  €, 

1=0  u=0  /=4} 


00  00 

«=o/=o 


(6.8) 
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where  (u'  =  1  +  u)  and  u'  >  I.  Accordingly,  if  we  set 

00 

~  ^u' t 

/=o 

Eq.  6.8  will  be 

00  00 

^C/  y,_/  =  ^  hu’  Xi_u‘  +  e,.  (6.9) 

1=0  u'=0 

Since  e,  is  a  pure  random  process,  Ci  and  hu-  can  be  estimated  by  the  least  squares 
method. 

From  Eq.  6.9 

00  00 

€,*  =  ^C*Yt-i-  ^  hu'X,^',  (6.10) 

1=0  u'=0 

(*  =  conjugate) 


and 

00  00 

€t+s  ~  ^  ^u'  (6.11) 

1=0  u’=0 

Taking  the  products  of  both  sides  of  Eqs.  6.10  and  6.11,  and  then  taking  the  statistical  ex¬ 
pectation  of  each  term 


OO  00  00  00 

=  Z  Z  C/  Ryyis)  ^  h,'  R^is  -  u' +  1) 

/=0  /=0  /=0  II '=0 

00  00 

-  ^  C;  hi'  RyAs-l  +  «') 

/=0  u'=0 

00  00 

+  Z  Z  (6.12) 

tt'=0 

Then,  by  the  Fourier  transform. 
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00 


(i')  e~^^do) 


i  I  R„(rt  e-^-du>  =  Xc;  e-'^^  j  R 

~ao  — QD 

-Xc;  X 

/=0  u  =l  ^  J 

-  X  C/  X  f 

/=0  tt'=0  ^  ■' 


If  we  set 


)  e 


,—i(S-u'-i-l)(l) 


da) 


\s-l+u')a) 


dCD 


+  X  hi-  X  f 

«'=o  «=o  ^  J 


ao 

XQ  =  CKo))  , 

1=0 

GO 

X  flu'  =  Hu«o)  , 

u'=0 

(6.13) 


(6.14) 


Eq.  6.13  becomes 


-^al  =  Ci{(oY:*iQ))Syy(w)-CKo>)HAQ>)Sxyio>) 

2jt  '' 

-Ciia))fCi(o)SyM 

+  Hu'{a))Hl'io))  ■  Syjjio)). 


(6.15) 


From  this  relation,  we  can  get  Sxr(<y).  %(<»).  andS;,y  (5y,),  and  Eq.  6.15  can  be  written 


(p- 

=  [Ci(fo),Hu<(o)\ 

2ji 


Syy^(0)  —  SxyifO  ) 

~  Syj^(0)  SxxO^) 


C/(a)) 

J 

tPu'ifo) 

(6.16) 


Equation  6.16  shows  the  relation  of  the  power  of  the  white  noise  to  the  spectra  of  input 
and  output  and  also  of  the  cross  spectra  of  input  to  output  and  the  frequency  response 
functions. 
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622  General  Two-Dimensional,  P-Dimensional  Case 

More  generally,  as  the  expansion  of  Eq.  6.8,  we  can  think  of  a  system,  diagrammed 
in  Fig.  6.3,  in  which  the  output  Xi,,  is  fed  back  to  the  input  A'2.,  as  well  as  the  input  Xz_, 
being  fed  to  the  output 


00  00 

■^1,/  =  X  X  ^1.2.u  ^Ij-u  ^  1,/ * 

/=!  i/=l 

00  00 

■^2./  =  X  ^2.1./  +  X  ^Ij-u  +  f2./- 

/=!  tt=l 


(6.17) 


U, 


A 

B 

X, 


u. 


Fig.  6.3.  Two-dimensional  feedback  system. 
This  equation  can  be  written  in  vector  form  as 


Cii.i 

C12.1 

Cii.2  C12.2 

Xu-2 

^2., 

C21.1 

C22.1 

X2J-1 

C212  C22.2 

X2,,.2 

m  m 

+  . 


Cnjt  Ci2jt 

Xu^ 

C2Uc  C22Ji 

X2,^ 

+ 


(6.18) 


Cutting  off  the  terms  at  appropriate  H:  by  some  criterion  gives 
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X,  =  a^Xi_^+  ^1-2'*'  ■  ■  ■  + 


k 

U=1 


(6.19) 


where  X, 


Cl\.m  Cl3jn 
CzA^  CzXm 


fl./ 

^2.1 


More  generally,  in  the  P-dimensional  process  the  dimensions  Xi.,,  Xz.,,  Xp  , 

are  connected  by  feedback  systems  to  each  other.  This  process  can  be  reduced  to  the  P- 
dimensionai  AR(/:)  model,  with  /:  as  an  appropriate  order. 


X,=  Aj  -^-1+  ^2  •  •  •  +  -^-*1:+  (6.20) 


■X,- 

^  1  l^m  •  •  •  C  \p^m 

fi' 

where  Xt  = 

Xz 

am- 

\ 

•  % 

•  t  . 

% 

f2 

% 

•  % 

% 

Eq.  6.20  can  then  be  written 


or 


x,=  2;  0.x  t-m  ^  t  j 
m=l 


l-X 

/n=l 


x.= 


or 


Q{B)  Xj  — 


(6.20’) 


(6.20”) 


here 


o(Z)  =  j^l  —  ^*2*1 


(6.21) 
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Then  the  spectnun  is 


Sxi(0} 


S(  {(O) 

\a{e-^f 


(6.22) 


2jc 


_ ^ 

1-  ai  e-^-a2  e-^  -  . 


-  Ok 


^-km 


(6.22  •) 


where 


€’] 


t  ^  U 

r  =  0  . 


(6.23) 


The  order  k  can  be  obtained  from  Akaike’s  minimum  AIC  criterion  in  this  case  too; 
i.e.,  we  can  find  k  that  minimizes 


AIC  ip, k)  =  iVlog \y  (I:)  +  Ip^k 


X. 


(6.24) 


where 


k 

m=0 


(6.25) 


as  it  was  for  the  scalar  case. 

Through  these  procedmes,  after  fixing  the  order  k,  we  get  the  coefficient 
a  1  ...  Ojt ,  as  for  the  scalar  case.  Then  we  can  get  the  spectrum  matrix  from  Eq.  6.22. 


SxjX^ 

SxyXi  ■ 

.  .  Sx,x^ 

SxiX2  • 

.  .  ^XiX^ 

• 

% 

% 

• 

> 

% 

%  • 

(6.26) 


The  cross  spectrum,  for  example  obtained  in  complex  form 

5x,^2(<u)  =  Cox^^ico)  +  iQxiXzifo)  (6.27) 

as  an  element  of  the  spectrum  matrix,  Eq.  6.26. 
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Then  the  frequency  response  function  of  jti  to  X2  is 


J\COx,x^(Oi)]^\Qu 

X\X2 

Sx^.iO}) 


|Gx,;t3(W)|  = 

The  phase  relation  is 

<t>x,xj<(i>)  =  Arg[G;c,x,(a>l]  =  tan 

and  the  coherency  function  is 


ij  QUx,xX(0) 

COx,x,(.lO) 


yW")  = 


Sx,x,{(0)Sx^^{(ii) 


(6.28) 


(6.29) 


(6.30) 


6.3  AUTOREGRESSIVE  CONTINUOUS  PROCESS 

So  far  only  discrete  processes  have  been  dealt  with  in  which  difference  equations 
have  been  used  to  formulate  the  processes.  When  we  study  the  response  of  some  dynamic 
systems,  however,  the  processes  are  in  many  cases  substantially  continuous.  Usually, 
however,  for  computational  analysis  these  processes  are  sampled  at  a  certain  time  inter¬ 
val  A/,  and  then  the  readings  are  digitized  and  are  treated  as  discrete  processes.  The 
analysis  technique  for  a  discrete  process  has  been  shown  in  detail  in  the  preceding 
sections. 

Sometimes,  however,  it  is  helpful  in  understanding  the  response  dynamics  of  a 
system  to  treat  the  process  directly  as  the  continuous  process. 

Although  difference  equations  have  been  used  to  formulate  discrete  processes,  dif¬ 
ferential  equations  are  used  to  formulate  continuous  processes.  We  are  more  accustomed 
to  dealing  with  differential  equations  to  express  the  physical  characteristics  of  the  re¬ 
sponse  of  the  dynamic  systems  than  with  difference  equations. 

In  this  section,  the  differential  equations  that  formulate  the  continuous  process  will 
be  introduced.  Then,  following  the  derivation  of  Pandit  and  Wu,^  the  relation  of  these 
differential  equations  to  the  difference  equations  which  formulate  the  digitized  processes 
will  be  smi^imarized,  since  in  practical  applications,  we  might  want  to  determine  the  char¬ 
acteristics  of  the  differential  equations  from  the  digitized  data  of  responses  and  inputs. 

63.1  The  First  Order  Continuous  Autoregressive  Process  A(l) 

For  the  first  order  continuous  autoregressive  process,  referred  to  as  A(l),  the  first 
order  differential  equation 

^Xit)  +  aaX{t)  =  Z{t)  (6.31) 

at 

can  be  formulated.  Here  Z(r)  is  the  forcing  function  or  the  white  noise,  expressed  as 
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E[Z{t)\  =  0 

£[Z(f)Z(/-i/)]  =  d{u)a\ 

=  £[Z(r)]2  =  (5(0)ai 


(6.32) 


5(0)  is  Dirac’s  delta  function. 


6{u)  = 


when  u  =  0 
when  u  ^  0, 


CO 

J 


and  I  6{u)du  =  1. 


(6.33) 


Accordingly,  £[Z(f)]'  =>  * ,  which  means  that  the  white  noise  Z(r)  is  physically 
unrealizable.  The  output  X^c)  is  a  stochastic  process  with  zero  mean.  Using  the  differen¬ 
tial  operator  D  =  d/dt,  D  =  {d/dtf, 

X(r)  =  (Z)  +  aor‘Z(r).  (6.34) 


When  we  express  Green’s  function  as  G(v), 


-\ 


X(r)  =  G(v)  •Z(r-v)dv  =  G{t-v)Z{v)dv, 


I 

=/ 


(6.35) 


This  equation  is  the  orthogonal  decomposition  of  X{t),  since  the  Z(f)’s  are  uncorre¬ 
lated,  or  independent  at  different  times.  From  Eq.  6.34  and  £q.  6.35,  it  is  also  clear  that, 
when  the  forcing  function  is  Dirac’s  delta  function,  X(f)  is  the  Green’s  function, 

(£>  +  cto)G(r)  =  (5(f) 


00 


G{t)=  I  G{v)d{t-v)dv. 


(6.36) 


Here,  since  the  solution  of  the  homogeneous  equation 

(£>  +  ao)-^X(f)  =  0 

is 

Xit)  =  C 


(6.37) 

(6.38) 


210 


where  X  is  the  root  of  the  characteristic  equation 

A  +ao  =  0,  or  A  =  -  flo, 

X{r)  =  C 


(6.39) 

'6.40( 


This  coefficient  C  is  determined  from  the  initial  conditions  given  by  the  charact  eris¬ 
tics  of  Dirac’s  delta  function  as  G(r)  =  0,  t  <0,  G(0)  =  1 ;  therefore  C  =  1 .  Green  s 
function  is  obtained  as 


G{n  = 


t  >  0 
0  ^  <  0 


(6,41) 


=  u(t) 


(6.41') 


here  u(r)  is  the  unit  step  function. 

The  autocorrelation  function  is^(s)  =  E[X{t)  X(r-  s)]. 


00 


Using .Y(f)  =  j  G(v')  Z(r-v')  dv' 
0 


00 

r-s)  =  1 1 


andY(r-s)=  |  G(v)  Z[,t-s-v)  dv 
0 

gives 

00  00 

i?(5)=  j  j  G{v')G{v)E[Z(t-v')Z{t-s-v)]dvdv' 

0  0 


00  00 

=  11  a^{s  +  v-v')G{v)G{v’)dvdv' 


0  0 


00  00 
=  I  G{v)G(v  +  s)dv  =  <^^e-°^^ 


R{-s)  =  Ris\ 


(6.42) 


thus 
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Now  suppose  we  have  the  sampled  discrete  data  from  a  continuous  process  with 
sampling  interval  Ar,  and  the  data  show  an  autoregressive  process  as  shown  by  Eq.  5.7’ 

X,-aiX,-i  =  €i. 

The  covariance  function  is,  from  Eq.  5.49,  ^(r)  =  — R{r)  =  (a|/l  -  a‘)a'. 

I  —a~ 

Relation  of  A(1 )  to  AR(1)  by  Covariance  Equivalence.  When  s  =  rAr,  the  auioco- 
variance  function  for  a  continuous  process  must  be  equivalent  to  that  of  the  discrete 
process  at  r  =  rAr.  Setting  s  =  rAr  in  Eq.  6.42  gives 

R(rAt)  =  (6.44) 

2ao  2cto ' 


For  discrete  computation  this  is  equivalent  io  R{r) 
Therefore 


or 


and 


^-aoAt  _  Q 

(6.45) 

Oi  _  Of 

2ao  l-a^ 

(6.46) 

=  lna  or  ao  = 

Ina 

(6.45’) 

Ar 

,  2aoa| 

(6.46’) 

From  these  relations,  we  can  convert  the  continuous  process  A(l)  into  a  sampled  process 
AR(1),  or  vice  versa,  by  inverting  the  coefficients  from  Eqs.  6.45’  and  6.46’.  From  Eq. 
6.43,  the  variance  is 

i?(0)  =  ^.  (6.47) 

2ao 


The  spectrum  is  given  by  the  Fourier  transform  of  /?(s)  as 


00 

=  fsie- 

2  1 
W  .-iws  1 

2jt  ] 

—  00 

[2a„  J 

1  1 
2jt  po  +  i(of  2ji  al  +  (o^' 

{ ■  }  showing  to  take  the  real  part  of  a  function  { - } . 

If  we  express 

(D  +  ao)  =  aiD), 


(6.48) 


then 


<77 

s(o})  =  — 


2ji  |a(Mo)p  ’ 


(6.49) 


which  is  again  a  form  similar  to  Eq.  S.S0  for  the  discrete  prcx^ess. 

632  Correspondence  ofA(2)  toARMA(2J) 

Generally  for  a  damped  mass  spring  linear  vibrating  system,  the  second  order  differ¬ 
ential  equation  as 

MX(t)  +  BX(t)  KX{t)  =  fif)  (6.50) 


stands  as  its  fundamental  formulation,  where  y(r)  represents  the  forcing  function. 
Here 

M  =  mass 

B  =  damping  coefficient  (linear  to  the  velocity) 

K  =  restoring  coefficient  (linear  to  the  displacement). 

Transforming  Eq.  6.50  by  conventional  orpressions  as 


gives 


or 


B 


1  M  _  ^ 

2  <o„  iIkM 


^Xit)  -h  2/c<o„-f  X(f)  wlX{t)  = 
dt  dt  M 


(6.51) 


(6.52) 
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UsLig  the  differential  operator  D-dfdt  gives 


(£)2  +  2A:a;^  +  to2)X(r)  =  4y(r),  (6.53) 

M 

or 

(D2  +  ajZ)  +  ao)X(r)  =  (6.53’) 

Af 

where  a\  =  2jca>„,  ao  =  u)\. 

Eq.6  .53’  is  analogous  to  the  AR  model  of  the  discrete  process  AR(2)  as  given  by 
Eq.  5.59  or  Eq.  5.62, 

+  02X1-2  —  ^  t  or  {I  +  OiB  +  02B^)Xt  =  €  i  ■ 


If  we  assume  {\/M)jf{t)  is  a  continuous  pure  random  process  Z(r),  then  we  can  say 
that  X(r)  is  an  autoregressive  process  of  continuous  time  of  order  2  and  express  this 
process  as  A(2)  as  did  Pandit  and  Wu.^ 

TTien  A(2)  can  be  expressed  as 


(D^  +  hcoJnD  +  (oi)X(,t)  -  Z(t) 


where  £[Z(r)Z(r')]  =  •  d(r-t')  =  j 


0 


when  r  =  r' 
when 


(6.54) 


which  corresponds  to  Eq.  5.62  for  AR  (2),  and 


X(t)  =  (I>2  +  2xa)„D  +  a)b~^Z(t)  =  (D^  +  aiD-h  ao)-^Z(r).  (6.55) 


Then  using  the  Green’s  function  G(r), 

X(t)  =  (D^  +  2x(o„D+Q)lr^Zit) 


go 


G(y)Z{t-v)dv, 


(6.56) 


and 

(£>2  +  aiD  +  ao)G{t)  =  <5(0  (6.57) 
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(6.58) 


OD 


G(r)  =  G(v)d(t-v)dv. 


Then  thinking  the  characters  of  derivatives  in  Eq.  6.57,  at  r  =  0,  G"(r)  that  is 
D^G{t)  contains  the  same  discontinuity  as  does  d(r).  Therefore,  G'(r)  must  contain  the 
same  discontinuity  as  does  the  imit  step  function,  and  similarly  G(t),  which  is  the  integral 
of  G'(r),  behaves  like  the  integral  of  the  unit  step  function,  which  is  a  ramp  function. 
Therefore,  G(t)  is  continuous  at  r  0,  and  the  initial  conditions  are 


G'(t)  =  G(t)  =  0,  t<0 


G'(0)  =  1,  G(0)  =  0. 


(6.59) 


From  these  initial  conditions  (Eq.  6.59),  the  coefficients  Ci,  C2  of  the  solution  of  the 


homogeneous  equation 

(D^  +  OiD  +  ctoKKO  =  0, 

(6.60) 

that  is,  of 

Git) 

(6.61) 

are  determined  as 

Ci=  — 

1 

i 

II 

1  < 
1 

(6.62) 

Therefore 

Git)  =  - 

”^2  Ai“A2 

— - - —  for  f  >  0 

X1-A2 

0  otherwise. 

(6.63) 

HereAi,A2  are  the  eigenvalues  or  the  roots  of  the  characteristic  equation  equated  to 
zero,  as 


f2iz)  =  Z^  +  aiZ  +  ao  = 

0. 

(6.64) 

Z^  +  2xa„Z+wl  =  (2-Ai)(Z- 

-A2)  =  0. 

(6.64’) 

Then 
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or 


(6.65) 


This  Green’s  function  is  the  so-called  unit  impulse  response  funcuon  that  shows  the 
response  of  this  physical  system  to  the  unit  impulse. 


Git)  = 


Aj— A2  Ai— A2 

This  impulse  response  function  shows  different  forms  as  follows,  depending  on 


(6.66) 


a.  When  /c  >  1  (erf  S  4cro),  A 1A2  are  real.  Therefore  G(r)  is  a  linear  combination 
of  two  exponentials,  as  is  shown  in  Fig.  6.4. 


6^2* 


Then 
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Using  G(t)  in  Eq.  6.56  gives 


Xit)  =  — - — Z(r-v)rfv 

J  A1-A2 
0 

t 

=  1  A. -A. 


(6.69) 


Equation  6.69  is  the  orthogonal  decomposition  of  X{t)  and  also  shows  the  general 
solution  of  the  nonhomogeneous  Eq.  6.54. 

The  autocovariance  function  is  then,  by  Eq.  6.56, 


R(s)  =  E[X{t)  X{t-s)] 


=  E 


*00 

00 

r 

G(y')Z{t-v’)dv' 

1  G(v)Z(t-s-v)dv 

J 

.0 

J 

0 

00  00 

=11 


G(v '  )Giv)EiZ(t  -v’)Z{t-s-  v)]dvdv ' 


0  0 


00  00 

1 1 


Giv''^v)d(y  +  s  -  v')dv' 


dv 


G{v)Giv  +  s)dv. 


(6.70) 


Inserting  in  Eq.  6.70  the  expression  G(v)  =  — - 5 —  and  manipulating  gives 

A\  *A2 


R(s)  = 


,2.  -f?-—  (6.71) 


UiiXl-Xl)  7X20.1- kl) 


Then  the  variance  is 
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^(0)  = 


-ari  ai 


21iA2(Ai+A2)  4Kaj^' 

The  autocovariance  coefficient  p(5)  is 

If  this  is  a  real  process,  then  Ji(-  s)  =  R(s)  and 

Therefore,  from  the  Fourier  uansform,  the  spectrum  is.  after  manipulation, 


s(m) 


R(.s)e-*^^ds 


■if 

.00 

~  7jt  ]  nMx\-xl) 


_£i 

"  2ji 


=  £i 


1 


(Kw)^  +  ai(Ky)  +  ao| 

_ 1 _ 


If  we  use  Eq.  6.54, 


2jt  ((a}^-(lin)  +  AK^(0ifJD 


(D^  +  Dai  +  OoJXit)  =  Z(f) 


a{Dmt)  =  m. 


(6.72) 


(6.72’) 


(6.71’) 


(6.73) 


Here  a(Z)  =  +  a\D + a©. 

Then  from  Eq.  6.73 


CT? 

sioi)  = 


2;r  |a(ia))p 


(6.73’) 


This  form  is  similar  to  Eq.  5.114,  which  is  the  expression  for  the  discrete  process 
AR(2). 
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Relation  ofA(2)  to  ARMA(2.1 )  by  Covariance  Equivalence.  As  was  discussed  in 
Section  5.3.5,  ARMA(2.1)  is  more  general  and  more  flexible  than  AR(2).  The  Green’s 

function  for  AR(2)  is  composed  of  two  exponentials  as  ^  step 

extension  or  complication  of  AR(1 )  process  where  the  Green’s  function  is  Gj  =  that  is 
one  exponential.  By  the  same  token,  we  can  say  that  ARMA(2.1)  corresponds  more 
closely  or  more  generally  and  uniquely  with  A(2)  than  AR(2)  does  with  A(2). 

Now  let  us  express  the  ARMA(2.1)  that  corresponds  with  A(2)  as  Eq.  5.120 


or  as  Eq.  5.123, 


X,-^  a\X^i  +  a2X^2  =  +  . 


(1  A  a\B  +  a2^^y^t  —  (1  +  b\B)€.t. 


The  autocovariance  function  for  this  ARMA(2.1)  is  from  Eq.  5.166, 

R{r)  =  Bifi\  +  B2M2^ 


where Hufi2  are  the  roots  of  the  characteristic  equation  equated  to  zero  for  ARMA(2.1). 
Here 

_/(Z)  =  +  a\Z  +  ^2  ~  0- 

From  Eqs.  5.67  and  5.75 


^1^2  =  02. 


± 


and  from  Eqs.  5.168  and  5.169 

d^ifi 


Bi  = 


ti  +  fri)  fii  +  h  fi2  +  bi 

-mf  1  -fi\  1 

■  m 


B2  = 


r 


ai(ji2  +  bi)  \fi2  +  bi  fii  +  bi 


(Ml -Mif  Y^-mI  1  - 


MiM2 


On  the  other  hand,  for  A(2),  when  s  =  rAr,  from  Eq.  6.71, 

4 


/?(rAr)  = 


2ii^2(Ai-Ai) 
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Therefore,  if  we  put  this  in  the  form 


=  d-2fi2, 

then 

d  ^ 

2X,{X\-Xl) 

d  -4 

2X2iXl-Xl)' 

Here  Ai.Ai  are  the  characteristic  roots  or  eigenvalues  of  Eq.  6.64.  Then 

Xi  +X2  —  —cci 
X1X2  =  oo- 

Then,  as  we  did  in  Section  6.3.1,  from  the  rule  of  covariance  equivalence 

thorefore  =  X 1 
A 

=  fi2  thwefore  =  X2, 

A 

and  also 

=  ;ti +/<2  =  - 


(6.74) 

(6.75) 

(6.76) 

(6.77) 

(6.78) 

(6.79) 


_  ^W,+A2)A/  =  fii/i2  =  02  therefore  In  <22  =  (^  1  +  A2)Ar  =  -  a  1  A/  (6.80) 


(6.81) 


Equating  the  values  of  di,d2  for  A(2)  as  in  Eq.  6.75  and  81,82  for  ARMA(2.1), 
expressed  by  Eqs.  5.168  and  5.169,  gives 
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_  +  hi) 

Mi  +  bi 

Ml  ^1 

7Xi(kl-Xl) 

(^1 

l-fii 

1  -M\M2 

Oz 

OeiMl  +  h) 

Ml  b\ 

M\-^bi 

2X2a\-xi) 

(Mi  -M2)‘ 

\-m\ 

l-MiMi 

Taking  the  ratio  on  both  sides  of  the  two  equations  in  Eq.  6.82  gives 

_ L _ . 

1  -fli  1  -/u^uj 

This  is  a  quadratic  equation  in  hi  and  has  the  form 

bl  +  2Pbi+l=0. 


After  manipulation,  we  get  . 


^  _  Aid  +^?)(1  +^2X1  -fA) 


and  also 


bi^-P  ± 

The  homogeneous  equations  of  Eqs.  6.53  or  6.54  is 

(Z>2  +  aiD  +  ao)X(f)  =  0 
(Z)2  +  2A:(u„  +  <oi)X(r)  =  0, 
and  its  eigenvalues  XuXi  are,  according  to  Eq.  6.65, 


(6.82) 


(6.83) 


(6.84) 


(6.85) 


(6.86) 


(6.87) 
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We  can  transform  Eq.  6.86  into  the  equation  expressed  by  ai,fl2.  and  bi  of  the  corre¬ 
sponding  ARMA(2.1)  model  by  the  relation  of  the  so-called  covariance  equivalence. 

a.  When  d\  <  4ao  («*  <  1) : 

Further,  for  simplicity  of  expression,  we  set 


Al  =  -  M  +  IV 

^2  =  -  w  -  iv. 


(6.88) 


Then 


V  =  ^j4ao-al  =  ^(o„^Jl-K^ 
1 

u  =  -ai 


(6.89) 


Cl  =  2u 

ao  =  u^  +  v^. 


(6.90) 


From  the  covariance  equivalence  relations  between  AR(2)  and  ARMA(2.1)  we  have 
Eqs.  6.77, 6.78, 6.79,  and  6.80.  Further  use  of  Eq.  6.88  gives 

-  ai  =  +^2  =  •  2  cos(vAr) 

fii -fi2  =  •  2i  sin  (vAr) 

1  -d(2  =  1  •  2  sinh  (uAr) 

1  +  <22  =  1  •  2  cosh  (mAt)  (6-91) 

fA~f*2~  2e~^^  •  I  sin  (2vAf) 

1  -<22  =  1  -  2e~^^  ■  sinh 


Inserting  these  expressions  into  Eq.  6.86  gives 


(6.92) 


V  sinh  (2uAt)-u  sin  (2vA;) 

2u  sin  (vAf)  cosh  (nAf)-2v  sinh  (uAr)  cos  (vAr) 


Then. 

u  [2P  sin  (vAf)  cosh  (ttA/)+  sin  (2vAf)] 

V  = - 

2P  sinh  (uA/)  cos  (vA/)+  sinh  (2uA/) 


From  Eqs.  6.90  and  6.81 


-  In  {aii  /— 


Therefore,  from  Eqs.  6.91  and  6.94 


cos  (vAr)  = 


-a\ 

2/^ 


■  /  A.. 

sm  (vAr)  = - y= - 

2  >102 


cosh  (wAr)  a 
sinh  (uAr)  = ' 


1  +  Q2 

2/^ 


(6.93) 


(6.94) 


^6.95) 


l^ith  Eqs.  6.95  and  6.85,  Eq.  6.93  can  be  transformed  into  an  equation  with  0|,  ai,  and  hi. 


V 


-  In  (02) 
2Ar 


7- (a? -4^2) 


-2ai  +  (l  +fl2)(^i  +'^) 
2(1  -a^)-ai(l  -a2)[bi  +-^) 


(6.96) 


U’  we  look  at  the  first  equation  of  Eq.  6.95,  v  is  derived  as 


(6.97) 


That  value  is  not  unique,  since  it  does  not  include  the  MA  parameter  bi.  However, 

Eq.  6.96  shows  that  v  is  uniquely  determined  from  a\,  02,  and  b\,  since  it  incluites  the 
MA  parameter  b\  as  well  as  a\  and  02.  This  is  another  reason  why  generally  we  prefer 
ARMA(2.1)  over  AR(2)  as  the  discrete  process  corresponding  to  A(2)  and  u  is  expressed 
byEq.  6.91. 
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After  u,  V  are  determined, a i.Cq,  or  R.w,  can  be  easily  derived  from  u,  v  by 
Eq.  6.90 

b.  When  a]  >  4ao  {<'  >  1) : 


We  can  set 

Aj  =  -«  + V 
A2  =  -tt  — V. 


(6.98) 


Then  computing  the  necessary  elements  for  P  in  Eq.  6.86  in  a  similar  way,  we  get 

■  2  cosh  (vA/) 

Hi  -Hi  =  •  2  sinh  (vA/) 

1  -H'^i  -  ■  2  sinh  (mA/) 

(6.99) 

1  •  2  cosh  (ttAr) 

iHi-^Hi)  0t^i-/^2)  =  2  sinh  (2vAf) 

(1  -ny^i)  =  2  sinh  (2u^ii). 

Inserting  these  values  in  Eq.  6.86  for  P  gives,  after  manipulation, 

p  _ _ u  sLnh  (2vAf)-v  sin  (2mA/) _ 

2u  sinh  (vAr)  cosh  («Af)— 2v  cosh  (vAr)  sinh  (mA/) 


Then 

u  [2P  sinh  (vAr)  cosh  (mAt)-  sinh  (2vA/)]  ^  „ 

v  = - .  (6.101) 

2P  cosh  (vAr)  sinh  («Ar)-  sinh  (2«A/) 


As  were  fora?  <  4ao,  the  sin,  cos,  sinh,  cosh  functions  are  expressed  by  aj,  ^2  for 
this  case.  Therefore,  after  v  and  u  have  been  determined,  a\,a2  or  k,  q}„  can  be  deter¬ 
mined  from  u,  V  easily.  Their  derivations  are  omitted  here  because  we  are  usually  less 
concerned  with  the  case  when  k^>  I  . 

For  this  case,  the  coefficients  ai,ao  or  k,  <o„  of  the  differential  equation  are 
derived  from  ai,  02.  hi  uniquely.  The  differential  equation  which  expresses  the  con¬ 
tinuous  autoregressive  process  can  be  obtained  from  the  parameters  ai,  a2,  and  hi  of  the 
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difference  eqixation  that  formulates  the  discrete  autoregressive  and  moving  average 
process  ARMA(2.1). 

The  above  mentioned  is  for  the  simple  case  of  ARMA(2.1)  versus  AR(2).  For  the 
more  comphcated  cases  of  ARMA(4.3),  ARMA(6.5) ....  theoretically  A(4), 

A(6), . . . ,  should  correspond,  and  the  parameters  of  differential  equations  that  formu¬ 
late  these  A(4),  A(6) . . .  should  be  derived  from  the  parameters  of  the  difference 
equations  that  formulate  ARMA(4.3),  ARMA(6.5) ....  although  their  relations  might 
be  much  more  complicated  than  for  A(2)  from  ARMA(2.1 ).  The  differential  equations 
that  formulate  the  continuous  autoregressive  process  give  us  good  clues  to  finding  the 
physical  characteristics  of  their  response  processes. 

The  second  order  linear  differential  equation  is  the  basic  equation  of  a  linear  dynam¬ 
ic  system  with  one  degree  of  freedom.  Accordingly,  the  above  discussion  indicates  that 
the  ARMA(2.1)  process  represents  the  dynamic  behavior  of  a  linear  system  with  one 
degree  of  freedom  under  the  excitement  of  white  noise. 
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CHAPTER? 


EXAMPLES  OF  MODEL  FITTING  TECHNIQUE  APPLIED  TO  THE 
ANALYSIS  OF  SEAKEEPING  DATA 

In  this  chapter,  examples  of  apphcation  of  the  modrtl  fitting  technique  to  seakeeping 
data  will  be  presented  to  demonstrate  the  apphcability  of  this  technique  to  the  analysis  of 
these  data. 

7.1  EXAMPLES  OF  AR(n)  MODEL  FITTING  FOR  THE  PREDICTION 

OF  SEAKEEPING  DATA 

Figure  7.1^  shows  examples  of  fitting  of  AR(n)  models  to  observed  seakeeping 
data.  After  the  appropriate  order  number  n  was  determined  by  the  MAIC  method,  as 
described  in  Section  5.5.4,  the  parameters  ...  a„  were  calculated  from  800  observed 
data  points  for  the  respective  processes.  Here  for  rolling  n=7  and  for  swaying  r7=19  were 
found  to  be  the  optimum.  Figure  7.1  shows  the  values  predicted  for  each  process  by 


Xt  = -a  iX,_i  -  a2X,.2 - a„K^„  (7.1) 


Fig.  7.1 .  Comparison  between  the  predicted  values  of  seakeeping  data 
and  the  observed  values. 

(From  Yamanouchi,  et  al.^) 
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using  these  ai  ...  a„.  One-step-abead  predictions  for  rolling  and  swaying  and  a 
10-  steps-ahead  prediction  for  swaying  are  shown  here. 

7.2  EXAMPLES  OF  MODEL  FITTING  TECHNIQUE  APPLIED 

TO  THE  SEAKEEPING  DATA 

Figure  7.2'*^  shows  examples  of  spectra  of  model  seakeeping  data  obtained  in  a 
model  basin  for  a  model  of  a  ship.  On  the  left-hand  side  (a),  the  spectra  calculated  by  the 
nonparametric  method,  the  so-called  Blackman-Tukey  method,  are  shown  and  on  the 
right-^iand  side  (b),  the  same  spectra  calculated  by  the  parametric  method,  i.e.,  the  AR(n) 
model  fitting  technique.  The  spectra  for  wave  height,  heaving,  and  relative  wave  height 
are  shown  on  a  logarithmic  scale,  and  the  Nyquist  frequency  of  2.50  shows 
Ar  =  (1/  2.50  X  2)  =  0.2  sec.  The  total  number  of  observations  is  AT  =  225,  which  is 

rather  small,  and  the  maximum  lag  number  for  the  B-T  method  is  50,  which  is  large  com¬ 
pared  to  N.  We  can  find  the  spectrum  from  the  AR  model,  the  order  of  each  process  being 
shown  by  the  AR  number  in  each  figure.  The  spectra  are  very  smooth  and  the  peaks  of 
the  spectra  are  sharply  defined  even  from  these  short  records  (small  number  of  observa¬ 
tions). 

Figure  7.3^  shows  the  behavior  of  AIC  values  that  were  used  to  find  the  order  n  of 
the  AR  models  fitted  to  each  set  of  these  seakeeping  data.  From  this  figure,  we  found  n  to 
be  11, 9,  and  10  for  wave  height,  heave,  and  relative  wave  height,  respectively.  These 
values  give  the  minimum  AIC. 

Figure  7.4  shows  the  time  series  of  an  AR(2)  model  simulated  by  the  difference 
equation, 

X,-0.5XM  +  0.7X,_2  =  e, 

when  ai--  0.5,  az  =  0.7  in  Eq.  5.59  for  a  general  AR(2)  model.  There,  e,  is  white  noise 
with  Gaussian  distribution  N(0, 1),  i.e.,  with  0  mean  and  a  variance  1.  The  time  series  are 
also  shown  in  the  same  figure.  The  number  of  observations  is  1,000  for  both  time 
series  €,  and  X,.  The  theoretical  spectrum  of  this  AR(2)  model  is  shown  at  the  top  of  Fig. 
7.5.^^  An  AR(n)  model  was  fitted  to  the  simulated  process,  and  from  AIC  criteria,  the 
optimum  n  was  estimated  as  n=2.  The  coefficients  a\,  az,  and  the  variance  a/  of  6,  were 
estimated  by  the  method  described  in  Section  5.2.3,  and  the  spectrum  ,r(o>)  was  calculated 
using  these  parameters,  as  shown  at  the  bottom  of  Fig.  7.5.  Independent  of  this  order  n=2 
by  minimum  AIC  criteria,  spectra  were  also  estimated  for  higher  n;  A(10)  and  AR(20) 
models  were  fitted  and  their  spectra  were  obtained  as  shown  in  the  same  Fig.  7.5. 
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Fig.  7.3.  Behavior  of  AlC  for  seakeeping  data  in  getting  the  spectra  (b)  in  Fig.  7.2. 

(From  Yamanouchi,  et  al.^) 


SIMULATION  DATA  N  -  1000 


WHITE  NOISE  €t 


Fig.  7.4.  Simulated  AR(2)  process  X,  =  0.5  Xm  -  0.7  X«  +  e„ 
Ef  is  a  white  noise  /V(0,1). 

(From  Yamanouchi,  et  al.®^) 
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DENSITY  IN  OB  DENSITY  IN  OB 


theoretical  spectrum  of  auto¬ 
regressive  MODEL  WITH  ORDER  2 


estimated  SPECTRUM  OF  SYNTHESIZED 
DATA  with  AR.2 


ESTIMATED  SPECTRUM  OF  SYNTHESIZED 
DATA  with  AR-2 


ESTIMATED  SPECTRUM  OF  SYNTHESIZED 
data  WITH  AR.2 


ESTIMATED  SPECTRUM  OF  SYNTHESIZED 
DATA  WITH  AR.2 


estimated  spectrum  of  synthesized 

DATA  with  AR-2 


(a)  AR  method  (optimum).  (b)  AR  method  (not  optimum).  (c)  B-T  method. 

Fig.  7.5  Comparison  of  AR-model  fitting  metfiod  and  B-T  method  and  the  effect  of 
changing  orders  of  AR-model  fitted  to  the  simulated  AR-2  process. 

(From  Yamanouchi,  et  al.®2) 
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For  comparison,  the  spectra  calculated  by  the  nonparametric  method  (Blackman- 
Tukey  method)  are  also  shown  in  this  figure.  The  maximum  lag  numbers  are  50  and  100, 
which  give  a  fairly  high  confident  estimate  since  these  numbers  are  1/20  and  1/10  of  the 
total  number  of  observations  /V=  1,000.  b  is  interesting  to  note  that  the  result  for  lag =50 
looks  much  like  the  one  for  AR(20).  The  impressive  point  here  is,  of  course,  that  the  esti¬ 
mate  from  the  model  fitting  technique,  which  determines  the  order  from  the  minimum 
AIC  criteria,  gives  a  result  that  checks  the  theoretical  process  very  well. 

Figure  7.6  compares  the  estimates  from  AR  model  fitting  and  the  B-T  method  for 
the  same  simulated  process  (Fig.  7.4)  for  a  wider  range  of  order  (AR  1-16)  and  maximum 
lag  numbers  of  20-500,  for  reference. 

Figure  7.7^^  shows  a  few  examples  of  the  analysis  of  actual  at-sea  ship  performance 
data.  Rolling,  wave  height  measured  by  buoy,  and  the  horizontal  acceleration  of  the  en¬ 
gine  bed  (as  of  different  frequency  characteristics,  because  the  engine  bed  is  mostly 
excited  by  the  prime  movers  of  the  ship)  were  analyzed  by  three  different  methods:  the 
AR  model  fitting  method,  the  F.F.T.  method,  and  the  Blackman-Tukey  method.  The  sam¬ 
pling  time  interval  was  Ar=  1  sec  for  roll  and  wave  height,  Ar=0.02  sec,  much  smaller  for 
acceleration.  The  spectrum  of  the  horizontal  acceleration  of  the  engine  bed  estimated  by 
the  AR  model  fitting  method  gives  very  reasonable  results,  compared  with  the  ones  ob¬ 
tained  by  other  methods,  the  former  clearly  showing  the  existence  of  multiple  natural 
frequencies. 

Figure  7.8^^  shows  other  examples  of  the  spectra  of  the  horizontal  acceleration  of 
the  engine  bed  estimated  by  the  AR  model  fitting  method  at  orders  n  of  5, 10, 15, 20,  30, 
and  55.  The  result  at  «=10,  where  the  AIC  shows  minimum  values,  looks  most  reasonable 
and  clearly  shows  the  existence  of  multiple  natural  frequencies  in  horizontal  vibration. 

7.3  EXAMPLES  OF  PARAMETRIC  ANALYSIS  OF  RESPONSE 

CHARACTERS  OF  MARINE  VEHICLES  AND  STRUCTURES 

Figure  7.9^^  shows  an  example  of  simple  response,  the  case  of  yaw  angle  versus 
rudder  angle  of  a  seagoing  ship.  The  results  for  the  spectra  of  input  and  output  and  the 
frequency  responses  from  two  methods,  the  AR  model  fitting  and  noiq)arametric  meth¬ 
ods,  are  compared.  When  the  spectrum  and  response  characteristics  do  not  show  any 
abrupt  changes  with  frequency,  as  in  the  example,  most  of  the  results  from  the  two  meth¬ 
ods  look  similar.  However,  when  we  look  at  the  Nyquist  diagram  or  the  mode  diagram  of 
the  frequency  response  characteristics,  the  results  from  the  AR  model  fitting  are  much 
better  and  smoother  and  more  reasonable,  physically,  than  those  from  the  nonparametric 
method. 

Figures  7.10  and  7.11^  show  the  results  of  seakeeping  data  for  a  model  ship  in 
waves  produced  in  the  towing  tank,  also  analyzed  by  two  different  methods. 

From  these  figures  we  see  that  the  AR-model  fitting  techniques  give  smoother 
curves  for  the  spectra  and  for  the  frequency  responses.  Also,  in  the  important  range  of 
frequencies,  the  coherency  values  are  closer  to  1,  showing  that  the  response  characteris¬ 
tics  are  more  reliable  when  estimated  by  this  method. 

In  both  figures,  the  cross  spectra  are  shown  by  their  real  and  imaginary  parts,  i.e., 
by  CO-  and  quadrature-spectra.  The  shape  of  these  spectra,  which  show  very  sharp  peaks. 
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(a)  AR  method.  (b)  B-T  method. 

Fig.  7.6.  Effect  of  changing  orders  in  AR-model  fitting  to  the  simuiated  AR(2} 
process  (Fig.  7.4)  [MAIC  shows  order  is  AR-2]  and  the  comparison 
with  spectrum  obtained  by  B-T  method. 

(From  Yamanouc^i,  et  al.®^ 
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(a)  AR  model  fitting. 


(b)  EF.T.  method. 


(c)  B-T  method. 


Rg.  7.7.  Comparison  of  the  spectrum  through  AR-model  fitting  and 
conventional  methods. 

(From  Yamanouchi,  et  al.52) 
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Fig.  7.8.  Effect  of  changing  the  orders  in  AR-ntodel  fitting  (by  MAIC;  AR-10). 
(From  Oda,  Yamanouchi,  et  2d.®®) 


indicates  that  the  AR  method  gives  better  results  than  the  B-T  method  and  more  precisely 
follows  the  rapid  changes  in  the  curves. 

By  the  way,  the  time  shift  techniques  that  this  author  proposed  in  Section  3.2  might 
have  given  even  greater  improvement  in  the  results  if  it  were  properly  applied.  The  rapid 
change  in  phase  response  with  frequency  shows  this  fact  and  indicates  t^t  there  is  still 
room  for  improvement  by  the  proper  shift  of  responses  even  in  the  case  of  using  model 
fitting  techniques.  Figure  7.12®^  is  an  example  of  the  same  kind  of  comparison  of  two 
analysis  methods  used  on  the  pitch  motion  of  a  model  of  an  offshore  semisubmersible  as 
shown  by  Fig.  7.13  on  irregular  waves  in  the  model  tank. 
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FREOUENCY  RESPONSE  ANALYSIS  BY  AR  MODEL 


(a)  Nonparametric  method  (max.  lag  =  50).  (b)  AR-model  fitting  method  (order  n=1 1 ). 

Fig.  7.10.  Comparison  of  methods  to  analyze  the  frequency  response  characters 
of  a  model  ship  in  the  tank  -  sample  1 . 

(From  Yamanouchi,  et  al.^) 


237 


PHASe  (0£G  )(XtO'|  OAJNjtNOBj  DfNSlIVIMOB 


POMfo  SPCCTAAl  oeNSiry 
cxjTPuT  :  ael  wave 


CAOSS  SAECTA*..  OENSro 
AEAi.  P«A7  I  t 


»(X««  SAECTAAw  OENSfT. 
OvTAw^  2  A£i  WiAVE 


CAO&S  SAECTA*.  3ENS 
AfeA.  AAA’  ■  2 


o«oso>oo  oxoi:'oc  oxcx’x  cxcs'C-x 


»A£OoENC»  fAEOuCNCV  fA£0»j£NC>  ‘AfcO^ENC 


CAOSS  SAEC7AA.  DEAfr'*'  AOwEB  SPECTRA.  DEhS  ^  CROSS  SAECTR-.  DeSS 


POWER  AND  CROSS  POWER  SPECTRUM 


POWER  AND  CROSS  POWER  SPECTRUM 


XAEOUENCV  RESPOASC  FUNCTION 

■NPcT  1  WAVE 

Output  2  AEL  WAVE 

N«32t 

H.SC 


FAEOUENCv 


FA£Ou£NC»  AESPOWSE  Fu^CtOA 
INP.'  ’  WAVE 
Ou’PU’  TAEL  WAVE 
A-22S 


FAEOOENCt 


FAEOUENCY  fAEOUEnCv  FR|OuENCv  FAEOUENCY 


FREQUENCY  RESPONSE  AND  COHERENCY  FREQUENCY  RESPONSE  AND  COHERENCY 


PAEOUENCV  RESPONSE  FUf.CTiON 

•♦PUT  1  WAVE 

OUTPUT  2  AEL  WAVE 

N-225 

H-M 


FREQUENCY  RESPONSE  ANALYSIS 


(a)  Nonparametric  method  (max.  lag  =  50). 
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FREQUENCY  RESPONSE  ANALYSIS  BY  AR  MODEL 

(b)  AR-model  fitting  method  (order  n=14). 


Fig.  7.11.  Comparison  of  methods  to  analyze  the  frequency  response  characters 
of  a  nfxxJel  ship  in  the  tank  -  sample  2. 

(From  Yamanouchi,  et  al.^) 
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(a;  Nonparametric  method  (max.  log  =  50).  (b)  AR-model  fitting  method  (order  n=8). 

Fig.  7.12.  Comparison  of  methods  to  analyze  the  frequency  response  characters  of 
a  model  offshore  structure.  (From  Yamanouchi,  et  al.®^) 


Fig.  7.13.  Photograph  of  a  semisubmersible  model  (scale  1/90). 

(From  Oda,  Yamanouchi,  et  al.®) 

Figures  7.14-7.17^^  show  the  results  of  frequency  response  analysis  of  the  model 
seakeeping  data  as  heave,  relative  wave  height,  pitch,  and  surge  in  irregular  waves  deter¬ 
mined  by  the  AR  model  fitting  technique  for  the  same  offshore  semisubmersible.  These 
data  were  analyzed  as  a  one-input/one-output  vector  process,  although,  if  we  like,  we 
can  analyze  them  as  a  one-input/multiple-output  vector  AR  process.  The  orders  of  AR 
models  fitted  to  these  systems  are  marked  on  each  of  the  figures. 

Figure  7.16  is  concerned  with  the  same  response  tmder  the  same  conditions  as  Fig. 
7.12.  From  these  results,  we  can  find  the  same  characteristics  of  AR-^nodel  fitting  tech¬ 
niques.  AU  the  spectra  and  the  frequency  responses  are  much  smoother  and  follow  well 
the  rapid  changes  with  the  frequency.  These  tendencies  are  more  clearly  indicated  by  Fig. 
7.18. 

In  Fig.  7.18  the  response  characteristics  of  heave,  relative  wave  height,  pitch,  and 
surge  are  shown  expanded  in  the  important  range  of  response  frequencies  for  6-24  sec¬ 
ond  periods  for  a  full-scale  structure.  The  results  obt'’ined  by  the  AR  model  fitting  and 
B-T  methods  are  shown  with  results  of  experiments  in  regular  waves  in  the  tank.  The 
figures  show  rather  surprisingly  how  well  the  results  from  the  AR  model  fitting  technique 
follow  the  minor  changes  of  the  response  in  amplitude  as  well  as  in  phase  relations  ob¬ 
tained  in  experiments  with  the  regular  harmonic  waves  of  several  frequencies. 
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Fig.  7.14.  Frequency  response  analysis  for  heave  by  AR-model  fitting 
(N=250,  Af=0.2  sec,  AR-8). 

(From  Oda,  Yamanouchi,  et  ai.®®) 
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Fig.  7.16.  Frequency  response  analysis  tor  pitch  by  AR-model 
fitting  {N=250,  At=0.2  sec,  AR-8). 

(From  Oda,  Yamanouchi,  et  al.®®) 
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Fig.  7.18.  Frequency  response  characters  by  different  methods  and 
through  experiments  in  regular  waves. 

(From  Oda,  Yamanouchi,  et  al.®^) 
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CHAPTERS 


CONCLUSION  FOR  PART  H 

In  Part  II,  we  have  discussed  the  model  fitting  technique  or  parametric  analysis 

method  for  a  random  process.  Comparing  this  method  with  those  discussed  in  Part  I  for 

nonparametric  analysis,  we  can  conclude  as  follows: 

1.  The  examples  in  Sections  52.1  to  5.2.5  show  that  the  autoregressive  (AR),  mov¬ 
ing  average  (MA),  or  autoregressive  moving  average  (ARMA)  model  of  rather 
low  order  can  represent  pretty  complicated  looking  processes  with  various 
frequency  characteristics. 

2.  For  a  given  process,  generally  we  can  fit  an  autoregressive  (AR),  moving  average 
(MA),  or  autoregressive  moving  average  mixture  type  (ARMA)  model  of  certain 
finite  order.  From  the  sample  autocovariance  functions  at  rather  low  lag  num¬ 
bers,  we  can  estimate  the  finite  number  of  parameters  of  the  model  and  the 
variance  of  the  residual  errors,  once  the  order  of  the  model  has  been  determined. 

3.  The  characteristics  of  the  discrete  parameter  model  are  closely  related  to  those  of 
the  correlations  of  that  model.  To  find  the  finite  number  of  parameters  to  express 
the  process,  the  correlation  function  until  the  finite  order  is  sufficient,  as  men¬ 
tioned  in  item  (2).  Sample  correlations  at  low  lag  numbers  are  more  reliable  than 
those  at  higher  lag  numbers,  and  there  is  no  need  to  be  concerned  about  the  lag 
windows  as  there  is  in  the  nonparametric  analysis  (the  so-called  Blackman- 
Tukey  method),  where,  ideally,  we  need  the  correlation  function  over 

-00  to  +  00  of  the  lag  numbers.  This  concern  about  lag  windows  and  consistent 
estimates,  resulted  in  many  efforts  to  find  good  windows.  Not  using  the  lag  win¬ 
dow  is  also  one  reason  why  we  can  get  better  results  by  the  model  fitting 
technique  then  from  rather  short  records. 

4.  The  order  of  die  AR(Jt),  MA(/),  or  ARMA(jfc,  /)  models  that  are  to  be  fitted  to  a 
sample  process,  as  the  best  from  a  statistical  point  of  view,  can  be  determined  by 
Akaike’s  information  criteria  (AIC)  as  the  combination  of  the  parameters  (k,  1) 
which  minimizes  AIC. 

5.  Evaluation  of  the  parameters  of  the  AR()t)  model  is  easily  performed  by  solving 
the  Yule  Walko*  equation,  which  is  a  linear  relation  of  parameters.  The  solution 
by  Yule  Walker’s  equation  is  a  good  approximation  of  the  solution  by  the  maxi¬ 
mum  likelihood  and  least  squares  mediod.  On  the  contrary,  for  the  and 
ARMA  models,  estimation  of  parameters  is  more  troublesome  and  difficult  and 
involves  solving  the  nonlinear  least  squares  equations. 

6.  As  far  as  stability  and  invertibility  are  fulfilled,  the  AR  and  ARMA  processes  of 
finite  order  can  ^  inverted  to  the  MA  process  of  infinite  order  and,  conversely, 
the  MA  and  ARMA  processes  of  finite  order  can  be  inverted  to  an  AR  model  of 
infinite  order.  Usually,  however,  in  this  approximation,  the  order  increases  from 
the  original  MA  or  ARMA  order.  The  order  of  the  AR  model  that  best  approxi¬ 
mates  the  original  MA  or  ARMA  is  obtained  through  the  AIC  crit^ia.  In  this 
approximation,  estimation  of  parameters  becomes  much  easier  than  for  the  origi¬ 
nal  MA  or  ARMA  process  as  mentioned  in  item  (5).  Accordingly,  AR  model 
fitting  is  usually  used  practically  for  general  processes. 
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7.  After  an  ^prophate  AR,  MA,  or  ARMA  model  has  been  fitted,  it  is  easy  to  cal¬ 
culate  its  spectrum.  The  spectrum  function  can  be  estimated  analytically  from 
only  a  few  parameters  that  characterize  the  fitted  model.  Hie  spectrum  thus  ob¬ 
tained  is  smooth  compared  to  the  spectrum  computed  by  the  nonparamethc 
method,  or  by  the  Fourier  transform  of  the  estimated  autocorrelation.  A  sharp- 
peaked  spectrum  can  be  obtained  by  this  method,  free  from  the  smoothing  or 
blurring  effects  of  the  spectrum  window  that  are  inevitably  applied  in  the 
nonparamethc  method. 

8.  From  the  charactehstics  of  the  model  fitting  technique,  we  can  get  a  fairly  reli¬ 
able  and  sharp  spectrum  even  from  rather  short  records  of  observation,  when 
reliable  autocorrelation  functions  at  large  lag  numbers  are  difficult  to  estimate. 

9.  The  AR  model  is  effective  in  finding  the  peak  frequency  and  in  estimating  the 
peak  value  of  the  spectrum.  Usually  the  second  order  of  an  autoregressive  model 
is  necessary  to  identify  one  peak  of  the  spectrum. 

10.  When  the  purpose  of  the  analysis  is  to  estimate  the  charactehstics  of  responses, 
the  AR  m^el  that  estimates  the  peak  effectively  is  an  approphate  approximation 
of  the  ARMA  model.  The  ARMA  model,  however,  is  more  reasonable  for  ex¬ 
pressing  the  response  of  a  linear  dynamic  process  to  random  excitations,  as  will 
be  mentioned  later. 

11.  To  express  the  flat  valley  of  a  spectrum,  however,  a  large  order  becomes  neces¬ 
sary  if  an  AR  model  is  to  be  fitted.  A  flat  valley  is  more  easily  expressed  by  an 
MA  model  of  low  order.  This  tendency  comes  from  the  character  of  the  trans¬ 
form  function  of  the  spectrum. 

12.  The  ARMA  model  that  expresses  peaks  by  an  AR  model  and  flat  valleys  by  an 
MA  will  be  the  most  appropriate  in  gener^  cases. 

13.  The  model  fitting  technique  relates  all  linear  stationary  processes  to  approphate 
AR,  MA,  or  ARMA  models.  All  of  these  models  assume  that  the  process  is  the 
output  of  a  pure  random  process  or  of  white  noise.  The  linear  relations  of  this 
process  to  white  noise  can  be  dehved  as  the  Green’s  function  of  these  models. 

14.  When  a  process  is  the  linear  output  of  some  input  that  is  not  necessarily  white 
but  colored,  we  can  get  the  response  of  this  process  to  the  real  white  noise  if  we 
assume  that  the  colored  input  is  the  output  of  the  real  white  noise.  In  this  way  we 
can  relate  all  the  colored  input  to  a  random  process. 

15.  Even  when  the  linear  respmise  system  has  smne  feedback  effects,  we  can  fit  a 
vector  autoregressive  process,  inverting  the  input  into  a  pure  random  process  by 
appl)dng  an  autoregressive  process  technique  to  the  input,  as  mentioned  in  item 
(14).  We  can  get  the  linear  response  characteristics  of  the  system  from  the 
elements  of  the  spectrum  matrix  of  the  vector  process.  In  observations  of  sea¬ 
keeping  data,  sometimes  the  feedback  effect  is  concealed,  so  this  method  can  be 
applied  effectively  in  the  analysis  of  such  data.  With  the  conventional  nonpara- 
metiic  method,  the  kind  of  system  that  has  feedback  effects  is  hard  to  handle. 

16.  A  second  order  autoregressive  continuous  process  A(2),  formulated  by  a  second 
order  linear  differential  equation  of  the  damped  mass  spring  vibration  type,  can 
be  transformed  into  a  discrete  ARMA(2.1)  process  formulated  by  a  difference 
equation  and  by  the  equivalent  correlation  dteory,  the  coefficient  of  the 
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differential  equation  can  be  expressed  by  tbe  coefficient  of  difference  equation 
and  visa  versa.  The  second  order  autoregressive  continuous  process  A(2)  is  then 
the  basic  process,  as  the  response  of  a  linear  vibration  system  to  the  white  noise 
with  one  degree  of  freedom.  Conversely,  when  a  dynamic  process  with  one 
degree  of  freedom  is  excited  by  white  noise,  the  digitized  response  can  be  ex¬ 
pressed  by  an  ARMA(2.1)  model.  When  tbe  input  is  colored  and  not  white,  we 
can  invert  this  system  to  a  pure  random  process  (white  noise  as  mentioned  in  item 
14)  although  the  order  increases.  Thus  &e  ARMA  model  is  the  most  general  pro¬ 
cess  for  er. pressing  the  oscillatory  response  of  a  linear  system. 

17.  An  advantage  of  the  parametric  analysis  is  that,  by  fitting  a  certain  discrete  model 
to  the  response  process,  theoretically  we  can  estimate  tbe  characteristics  of  the 
differentik  equation  that  governs  the  response  of  a  continuous  process  by  invert¬ 
ing  the  difference  equation  that  expresses  the  discrete  model  into  tbe  equivalent 
differential  equation  that  formulates  the  continuous  process. 

1 8.  The  confidence  limit  of  each  parameter  of  a  fitted  discrete  model  can  be  eva¬ 
luated.  The  remaining  statistical  considerations  are  taken  care  of  by  adopting  the 
most  adequate  order  of  the  model  based  on  Akaike’s  information  criteria.  How¬ 
ever,  if  we  could  express  the  overall  reliability  by  some  simple  expression,  say, 
for  example  by  a  confidence  band  at  some  level  of  probabihty,  it  would  be  easier 
to  give  us  more  confidence  in  the  parametric  method,  directly  comparing  with  tbe 
result  by  the  noiqiarametric  method. 

19.  The  AR  model  was  adopted  here  first  and  was  applied  to  the  seakeeping  data  of 
ships  and  offshore  structures,  both  for  models  in  &e  tank  and  for  the  actual  ships 
and  structures  at  sea.  From  the  results,  we  found  this  model  fitting  technique  was 
practical  to  apply  and  was  a  promising  metnod  of  analysis,  sometimes  giving  us 
better  results  th^  the  conventional  nonparametric  method.  At  least  this  model 
fitting  or  parametric  analysis  technique  si^plements  tbe  nonparametric  method. 

20.  The  ARMA  model  is  the  most  general  and  is  directly  connected  with  the  continu¬ 
ous  AR  model  or  its  formulating  differential  equations  that  govon  the  behavior 
of  the  system.  It  seems  to  be  the  most  desirable  for  estimating  the  physical  char¬ 
acteristics  of  the  process,  although  its  handling  m  the  estimation  of  parameters  is 
more  complicated  than  for  a  pure  AR  model.  Accumulation  of  experience  with 
this  triplication  is  now  very  much  needed,  this  author  believes. 
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APPENDIX  A1 

DATA  FOR  THE  GENERATION  OF  THE  PROCESSES 

Data  for  the  generation  of  a  pure  random  process  AR(1),  AR(2),  ARMA(2.1), 
MA(2),  MA(1),  and  ARMA(2.2).  given  in  Chapter  5,  Figs.  5.3, 5.10. 5.20, 5.25, 5.28, 
5.32, 5.35, 5.38,  and  5.41,  and  Tables  5.1  and  5.2  are  listed  on  the  following  pages. 
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APPENDIX  A2 


POLYNOMIAL  MODEL  FITTING  TO  OBSERVED  DATA-^ 

Given  pairs  of  data  ( A,,  } ,  j  =  1  •  •  •  N,  we  often  want  to  express  Y,  as  a 

polynomial  of  Xj. 

If  we  fit  yM.i,  a  polynomial  of  jc,  of  order  M,  as 

yM-i  =  ao  +  aiXi  +  a24  +  -  ■  +0/^  (A2.1) 

/  =  1,-  •  -N, 

we  can  estimate  the  coefficients  ao,  ai  •  •  •  a*,  by  the  least  squares  method.  Namely, 
computing  the  mean  square  of  the  difference 

^  w  =  SO'/  -  (^-2) 

and  fmd  the  ao,  ai  •  •  •  am  that  makea^  minimum.  Then  from 


da 


dao 


da  ^ 


-  =  0,  ao+{j^xi)ai  +  -  •  •  +  (X^)^A^= 


M 


dOi 


= 0,  (X  ^«)«o + (X  + •  •  • + (X  ^ 


=  0.  (X;cf)ao  +  (X4^iai^-  •  -h(X^K=X^-^.  - 


(A2.3) 


The  ao  -  -  -  are  calculated  as  the  solution  of  the  simultaneous  Eq.  A2.3. 

Here,  the  most  serious  problem  is  determining  the  order.  We  can  make  y^i  as  close 
as  we  wish  to  y,  by  making  the  order  M  very  large.  However,  if  Af  is  too  large,  ymi 
follows,  even  to  the  random  error  of  the  observations.  Of  course,  if  M  is  too  small,  y^i 
sometimes  neglects  the  variations  in  the  data  that  really  exist,  as  shown  in  Fig.  A2.1 . 
Order  1  is  too  small,  but  order  5  seems  too  large. 
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ORDER  1 


ORDER  2 


Rg.  A2.1 .  Fitting  of  polynomials  to  the  observed  data. 

(From  Yamanouchi,  et  al.^®) 

All  observed  data  include  the  statistical  error.  Accordingly,  when  x,  are  given, 
instead  of  Eq.  A2.1,  y,  should  be  represented  by  a  regressive  polynomial, 

yi-ao-^aiXi  +^24  +  -  •  •  +  +  (A2.4) 

Herec,-  is  a  probability  variable  that  follows  the  normal  distribution  with  0  mean  and 

variance  a}.  Here,  we  can  define  that  model  fitting  is  the  statistical  selection  of  a  model 
to  fit  best  to  data  in  statistical  meaning.  When  we  use  Eq.  A2.4  as  the  model,  we  can  use 
the  minimum  AIC  criteria  or  the  MAIC  method  to  determine  the  order  Af .  When  y,  is 
approximated  by  a  polynomial  of  order  Af  as  m  Eq.  A2.1,  the  coeSicienta/s  has  the 
values  determined  by  the  least  squares  method, 

N 

- L  =  inin( l/N)  ^  (y,  - (A2.5) 
«  =  l 

Then  let  us  investigate  the  behavior  of  this  mean  square  of  the  residual  error 

i=l 

or  N  times  the  logarithm  of  Eq.  A2.5 

AFlog5](y,  -yJ^f  ,)2=Arioga  (A2.6) 

This  value  A2.6  is  shown  in  Fig.  A2.2  by  the  mark  O  and  tells  us  that  the  larger  the  value 
of  Af,  die  smaller  this  value  A2.6  is,  or  the  larger  the  order  Af,  the  smaller  the  residual 

error  al 
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Fig.  A2.2.  Behavior  of  AlC  for  Fig.  A2. 1 . 
(From  Yamanouchi,  et  al.^) 


However,  this  procedure  of  guiding  M  from  Eq.  A2.6  does  not  give  us  the  best 
values  of  the  order,  because  Eq.  A2.6  does  not  include  the  goodness  of  fit  to  the  real 
structure  of  the  data  or  the  decrease  in  reliability  with  the  increase  in  the  number  of 
parameters  to  be  estimated.  So,  to  show  the  penalty  for  this  increase  in  the  order,  if  we 
add  the  term 


2  X  (no.  of  parameters)  =  2(M  +  2)  (A2.7) 

to  Eq.  A2.6,  that  gives  us  the  AIC, 


AdCiM)  =  N log  (yi  - yMjf  +  2(Af  +  2).  (A2.8) 

i=  1 

1  he  number  of  parameters  M +  2  comes  from  (Af  + 1)  a/s  for  ao,a\  ■  an  plus  1 

for  <7  ^  that  we  must  also  estimate.  The  behavior  of  2(Af  +  2)  is  shown  in  Fig.  A2.2  by 
the  mark  0  as  a  straight  curve  that  increases  linearly  by  Af.  Then  AIC,  expressed  as  the 

sum  of  N  log  a  ^  and  2  (Af  +  2),  behaves  like  the  ^  in  Fig.  A2.2  and  shows  a  TninimiiTn  at 
a  certain  AY,  here  Af  +  1  =  3  for  the  data  shown  in  Fig.  A2.1.  Thus  the  order  of  the 
polynomial  is  determined  as  Af  =  2  by  AIC  criteria. 
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From  the  discussions  in  Section  5.4.3,  we  know  that  Eq.  A2.5,  the  estimation  that 

N 

minimizes  (1 /AO  is  the  estimate  that  maximizes  the  likelihood  function!., 

is] 

or  the  logarithmic  likelihood  function  log  L.  Accordingly,  Eq.  A2.8  can  be  written  as 

AIC  =  -  2 log(inax  likelihood)  +  2(no.  of  parameters).  (A2.9) 

Eq.  A2.9  supplements  the  characteristics  of  AIC,  this  author  believes.  As  another 
example  of  polynomial  fitting.  Fig.  A2.3  shows  the  plot  of  the  observed  data  for  a 
maneuverability  test  of  a  ship. 

By  the  same  procedure,  from  the  behavior  of  AIC  in  Fig.  A2.4,  the  order  was 
determined  as  Af  =  5,  and  the  fifth  order  polynomial  was  estimated  as  the  best  fit  to  these 
data. 


Fig.  A2.3.  Fitting  of  poiynomials  to  the  data  of  maneuvering  test  of  ship. 
(From  Yamanouchi,  et  al.*®) 
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On  Fig.  A2.4,  the  behavior  of  AIC  is  shown  for  this  fitting. 


Fig.  A2.4.  Behavior  of  AIC  for  Fig.  A2.3. 
(From  Yamanouchi,  et  al.^) 
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PART  in 


TREATMENT  OF  NONLINEARITIES 

CHAPTER  9 
INTRODUCTION 
9.1  INTRODUCTION  FOR  PART  m 

In  the  preceding  sections.  Part  I  and  Part  H,  the  processes  were  assumed  to  be  al¬ 
most  linear,  and  in  so  far  as  the  process  is  linear,  the  pehodogram  analysis  shown  in  Part  1 
and  the  model  fitting  method  in  Part  n  are  effective  methods  of  analysis.  For  example,  a 
Founer-Stieltjes  expression  for  the  process  assumed  to  exist  in  Part  1  is  originally  a  linear 
expression,  and  the  AR,  MA,  or  ARMA  models  treated  in  Part  II  are  based  on  the  idea  of 
decomposing  the  processes  into  independent  or  orthogonal  processes,  i.e.,  assuming  their 
linearity.  However,  if  the  process  is  nonlinear,  these  methods  can  no  longer  be  applied 
directly.  We  need  some  special  considerations  for  their  treatment 

In  most  engineering  fields,  many  phenomena  can  be  approximated  as  linear.  How¬ 
ever,  no  phenomena  are  purely  linear  but  include  some  elements  of  nonlinearity.  Today  in 
nonlinear  phenomena  such  as  the  effect  of  viscosity,  the  secondary  potential  forces  that 
include  the  interaction  of  two  frequency  components  of  excitations  have  come  to  be  con¬ 
sidered  important  in  treating  the  behavior  of  ocean  vehicles  and  structures,  although 
useful  information  has  been  derived  even  under  the  limitations  of  linear  approximation. 

Here  in  Part  HI,  the  nonlinearity  of  waves  themselves  that  are  the  source  of  excita¬ 
tion  to  systems  of  ocean  vehicles  and  structures  is  first  investigated.  According  to  a  few 
works  already  published,  their  nonlinearity  is  usually  not  large. 

The  ap^H-oximation  methods  that  have  been  used  to  treat  nonlinear  response  pro¬ 
cesses,  such  as  the  linearization  method  and  the  perturbation  method,  are  summarized.  As 
more  advanced  approximations,  the  Voltera  or  functional  expansion  of  the  nonlinear  pro¬ 
cess  and  the  apphcadon  of  polyspectra  are  then  introduced. 

As  an  extension  of  these  approximation  method,  a  shghtly  different  aspect  of  the 
analysis  of  the  probability  characteristics  of  the  process,  the  probability  distribution  func¬ 
tion  of  the  extremes,  is  summarized.  As  extensive  work  has  been  published  recently  on 
the  analysis  of  nonlinearity  response  including  these  probability  characteristics  by  J.  F. 
Dalzell,*^’*^’^^^  only  the  derivation  of  general  characteristics  of  several  functions  and 
the  results  of  these  applications  are  reviewed. 

Next  in  Part  ni,  the  treatment  of  stochastic  processes  as  Markov  processes  is  intro¬ 
duced,  and  then  the  application  of  the  Fokker-Planck  equation,  also  recently  introduced 
by  J.  B.  Roberts^^*  in  the  analysis  of  seakeeping  data,  is  reviewed. 

Finally,  as  a  slightly  different  approach,  or  as  the  extension  of  the  model  fitting  tech¬ 
niques  discussed  in  Part  n,  a  few  examples  of  this  extension  to  the  nonlinear  process  are 
briefly  reviewed. 
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9.2  NONLINEARITY  OF  OCEAN  WAVES 

Wave  theory  has  developed  remarkably  in  recent  decades  with  the  assumption  of  a 
wave  as  a  stochastic  process.  In  most  cases,  however,  the  assumption  is  based  on  infmi- 
tesimal  amplimdes  and  is  valid  only  when  the  wave  height  is  small  compared  with  the 
wave  length,  and  wave  length  is  small  compared  with  the  water  depth.  All  the  quadratic 
terms  of  the  derivatives  were  assumed  to  be  small  and  were  neglected,  and  the 
fundamental  equations  of  motion  were  linearized  and  the  linearized  potential  function 
was  derived. 

In  this  section,  the  results  of  only  a  few  works  on  the  investigation  of  the  effect  of 
this  neglect  will  be  reviewed  to  show  that  the  effects  are  indeed  small. 

92.1  Second  Order  Specrrum  of  Waves  by  L.  J.  Tick  and  Others 

To  take  account  of  quadratic  terms,  L.  J.  Tick^-’^^  expanded  the  potential  function  of 
a  wave  <p(x,  z,  t)  around  the  mean  position  of  the  wave  r  =  0  by  Taylor  expansion. 


2=0 


dz 


z=0 


z+  .  .  .  , 


(9.1) 


and  carried  it  to  the  second  term.  Using  this  expression  in  the  fundamental  equations  for 
waves,  through  the  perturbation  method  he  expressed  this  velocity  potential  as  the  sum  of 
the  linear  part^](r)  and  the  quadratic  term ^2(0  ^ 

^(0  =  ^i(0  +  ^2(0.  (9-2) 

As  a  result,  the  wave  height  is  supposed  to  be  composed  of  two  parts,  linear  and 
quadratic,  corresponding  to  these  two  potentials,  as 


and  expressed 


Here 


£[ld^P]  =  :f\w)(ko. 


(9.3) 


(9.4) 


(9.5) 
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00  00 

=  I  j  Q(Q),0)')d^{(o)d^{Q)’),  (9.6) 

<-00  —00 

and 

Q((0,a}')  =  -  [5gn(aj),:rg/i(ct;*)]  ^  ■  (9  '7) 

Then  the  spectrum  of  t]{t)  was  derived  as 


—  00 


=  +  (9.8) 

r 

As  an  example,  Tick  took  the  simplified  Neumann-Pierson  type  spectrum 
asj^^^o)  =  1.8  X  l.0*Q)'*kt>\  >  0)0  and  computed  this  s^^\(o)  fortuo  =  2,3,6  rad/sec. 

Figure  9.1  shows  the  results  and  (l/g^>s<^\a))  is  as  small  as  several  hundredths  of 
s^^\o))  at  the  most. 

L.  Hck*^  considered  also  a  shallow  water  depth  and  derived  the  type  of 
Qh{o),o)')  for  this  case  that  corresponds  to  Qi(o,0)')  in  Eq.  9.6,  as 


Qh(0},Q)')  = 


1  \k  k'\k  k'  CD  (o'  (o+o)')^ 


(O  CD 


(CD  +  CD 


>^2r  Ikk'lkk'  1 

\  WO)'  2 


flJ  <0  + 


w  k'*-Ho'  k* 
2a  w'(w+w') 


^  J 


(l/d  it  +  lfc'l  k)  tanh  [d\\ld  k+\ld  Jt')]  -  (oi  +  cu') 


'^2 


(9.9) 


Here  if  =  (h/g),  k  =  k(o)),  k’  =  /fc(a>')  are  the  solutions  of  o)^  =  if  tanh  dfif.  As  an 
example,  he  showed  P\(o)  of  s(<u)  =  for  the  following  type  spectrum. 
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82(0))  for  8,(0))  -  18  X  1 0',  |0)|  >  0)o 

Fig.  9.1.  The  second  order  spectra. 

(From  Tick.®2) 

=  C  (o-^  e-^,  (o<2 

and  showed  as  in  Fig.  9.2.  In  this  case,  even  when  the  water  depth  h  =  32.2  ft,  the 

effect  of  nonlinearity  is  small. 

He  pointed  out  the  necessity  for  computing  the  bispectrum  to  show  clearly  the 
quadratic  effect  of  the  waves  as  will  be  discussed  in  Section  9.2.2. 

On  the  nonlinearity  of  the  waves  M.  S.  Longuet-Higgins*^’  and  D.  M.  PhilUps“ 
discussed  work  along  the  same  lines,  and  M.  Hineno^^  referred  these  works  and  showed 
clearly  the  quadratic  effect,  as  in  Fig.  9.3,  that  was  calculated  for  a  modified  Moskowitz- 
Pierson  type  spectrum.  In  Fig.  9.4,  the  quadratic  and  linear  parts  of  the  spectrum 
produced  in  the  experimental  water  tank  are  shown. 

Hineno^^  also  showed  the  quadratic  kernel  functions g2{T\,ri)  as  Fig.  9.5,  that  is,  the 
Fourier  transform  of  quadratic  frequency  responses  G2(tOi,m2),  equivalent  to  the 
Q(o),a)')  byL.  Tick,  as 
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Rg.9.2.  The  second  order  spectra  /^Ka>)  for  varying  depth  h  =  32.2, 46.4,  and  72.5  ft. 

(From  Tick.®3) 


1  O  T 

GiCoj  1 , 0)2)  =  —  (.0)1+  0)2)  for  the  sum  (o)  1  +  0)2)  frequency  component 

+8 


loo 

=  -—  IrWi-oll  for  the  difference  frequency  component.  (9.10) 

2g 


Figures  9.3  and  9.4  indicate  that  the  quadratic  effect  on  the  spectrum  computations 
is  very  small  for  ordinary  cases.  M.  Hineno®^  extended  this  result  and  calculated  the  prob¬ 
ability  distribution  of  the  maximum  and  minimiim  of  the  waves.  The  results  are  intro¬ 
duced  in  Section  12.7,  and  from  the  results  we  find  that  the  effect  of  the  nonlinearity  is 
not  large. 

92-2  Bispectrum  of  Waves 

L.  Tick®^  suggested  the  need  for  bispectrum  analysis  for  ocean  waves.  Such  an  anal¬ 
ysis  was  performed  by  K.  Hasselman  et  al.^®  as  shown  in  Fig.  9.6.  The  exposition  on  the 
bispectrum  is  given  in  Section  11.3.  Here,  one  bispectrum  for  shallow  waves  is  given  as 
an  example,  and  it  indicates  that  the  nonlinearity  is  not  large  in  this  analysis  either. 
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Fig.  9.3.  Linear  and  quadratic  component  of  wave  spectrum  of  Pierson-Moskowitz. 


Rg.  9.4.  Linear  and  quadratic  component  of  wave  spectrum  of 
waves  produced  in  a  towing  tank. 

(From  Hineno.®^) 
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Fig.  9.6.  Bispectrum  of  ocean  wave. 

[The  number  -7^  denotes  -7x10^  cm®  sec^.  Contours  are  drawn 
for  -10®,  -10^,  -10®,  and  -10®.]  (From  Hasselman,  et  al.®®) 
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9.3  RESPONSE  OF  THE  BEHAVIOR  OF  A  MARINE  VEHICLE  ON  WAVES 

When  the  ana^phtude  of  oscillation  in  six  degrees  of  freedom  is  small,  the  behavior 
of  a  marine  vehicle  in  waves  can  be  expressed  approximately  by  linear  equations. 

Here,  for  simplicity,  if  we  assume  that  a  floating  body  in  a  regular  wave  train 
5(r)  =  Z{a))e^'  is  subjected  to  a  force  b^(t)  that  is  linear  with  wave  height  ^(r),  the 
equation  of  motion  is  well  approximated*’  by 


aix(t)  +  a2x{t)  +  a^xit)  =  bt(t). 


(9.11) 


The  response  jc(r)  will  be  in  the  formX(<u)e^', 
where 


Xio)  (9.12) 


d^ioXOiiCD)  are  the  phase  relations  of  the  exciting  force  to  the  wave  height,  and  the 
response  to  the  exciting  force,  respectively. 

Generally,  for  a  body  floating  on  the  surface  of  water,  the  coefficients  ai,a2,bof 
the  inertia  term  aii(r),  the  damping  term,  and  the  compulsory  term,  respectively,  are 
generally  functions  of  the  frequency  and  so  we  find  oi  =  Ai(a)),a2  ~  Azftu),  and 
as  =  As((o),  b  =  B((w), 

The  equation  of  motion,  Eq.  9,11,  will  be 


-a}%((o)X(co)e^'  +  j(oA2{a))X{a))e^‘  +  A3(a»)X(m)e^'  =  B{(o)Z{fo)e^\  (9,13) 


Therefore,  the  frequency  response  function  Hx^o>)  of  the  behavior  X(tu)  to  wave  height 
2(0))  is 


Xja)) _ £(0)) _ 

Z(g))  -Q}^Ai(o>)+j(oA2i(o)+As{Q)) 


(9.14) 


From  this,  expressing  Hx^co)  just  as  H((o), 
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X((i})  =  H{(o)Z(o}). 
Taking  the  Fourier  transform  gives 


(9.15) 


xir)  = 


1 


-00 


hir%(t-x)dT. 


(9.16) 


Here 


Kx)  = 


1 


H(a))e^d(o, 


(9.17) 


is  the  impulse  response  function. 

When  the  waves  are  expressed  by  a  Fourier  integral, 

00 

5(f)  =  ^  j 

—  00 

the  linear  response  x(r)  will  be 

00 

x(f)  =  —  I  X{(o)^’(ko. 
2jt  J 
—  00 


(9.18) 


(9.19) 


Strictly  speaking,  these  integrals  (Eqs.  9.18  and  9.19)  should  be  Fourier-Stieltjes 
integrals  in  the  form  of 


(9.20) 


where 
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dZicD)  dZ{(o')  =  Sz{<i})di(a +a:') 
dX{(o)  dX.ip}')  =  +  o') 


(9.21) 


Siio)),  Sj^Q})  being  the  spectrum  of  wave  height  ^(r)  and  response  x(r),  respectively.  Here, 
however,  for  simphcity  we  use  the  ordinary  Fourier  integral  form. 

From  Eqs.  9.18, 9.19,  and  9.13,  Eq.  9.11  will  be 


J_ 

2jt 


00  00  00  * 

j  |-a)^Ai(m)|  X(,(i))e^‘da)+  |  [iQ)A2(o})]X{Q})e^'dQ)-^-  J  Aii(o)X(Q})e^'dQ) 


2jt 


I 


^ec 


BiQ))Z{(o)e^‘d(0. 


(9.22) 


From  Eq.  9.19 


Then  setting 


xit-t) 


00 


X{o))e^U-^do). 


~ao 


oo 


1 

~Q0 

I 


A,iQ))e^dm  =  fc,<T) 


B(fi))e^do}  =  /(t). 


Eq.  9.22  will  be  in  the  form  of  a  Fourier  convolution. 


(9.23) 


(9.24) 
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ki(j:)xit-r)dr  + 


I 

~00 


Ic2(T)Mf-T)dT^  k3(T)x(r-T)dr 


I 


ac 

-I 


lit)  z(r-T). 


(9.25) 


Actually,  however,  taking  into  account  the  Kramer-Kronig's  theory  that  connects 
the  added  mass  and  the  damping  for  floating  and  oscillating  bodies  and  also  the  fact  that 
the  restoring  term  a3X(t)  is  not  a  function  of  frequency,  the  equation  of  motion  is,  as 
T.  F.  Ogilvie™  pointed  out,  in  the  form  of 


(Af  +  m(  00 


K(t-T)  xit)dt  +  C 


(9.26) 


Here  m(ao)  is  the  added  mass  at  frequency  m  =  ®. 

9.4  NObO^IbnEARITY  OF  THE  BEHAVIOR  OF  MARINE  VEHICLES 

When  the  waves  are  moderate  and  the  amplitudes  of  the  motions  are  mild,  the  re¬ 
sponse  of  marine  vehicles  is  well  expressed  by  linear  equations,  as  was  shown  in  the 
preceding  sections. 

The  nonlinearity  of  ocean  waves  was  found  to  be  not  as  large  as  was  shown  in  Sec¬ 
tion  9.2.  Even  when  the  wave  itself  is  linear,  sometimes  the  exciting  force  exerted  by  the 
wave  can  be  nonlinear  with  respect  to  wave  height  For  example,  for  drifting  vehicles  or 
for  the  slowly  varying  behavior  of  moored  offshore  structures,  the  exciting  force  is  not 
linear  with  the  wave  height  because  of  the  effects  of  the  secondary  potentials  of  waves. 
For  rolling  near  the  synchronous  frequency,  the  vehicle  might  oscillate  with  such  a  large 
amplitude  that  the  damping  and  the  restoring  force  are  not  linear  with  the  velocity  or  the 
displacement.  For  oscillatory  motions,  in  many  cases  viscous  resistance  that  is  not  linear 
to  the  velocity  of  motion  sometimes  plays  a  big  role  in  addition  to  that  of  wave  making 
resistance  that  is  linear,  and  results  as  nonlinear  damping. 

Usually,  however,  even  in  cases  of  this  kind,  the  effect  of  nonlinearity  is  assumed  to 
be  small  or  weak  in  the  succeeding  discussions,  as  will  be  mentioned  again  in  (Thapter  11. 
Generally,  under  certain  restrictions  (as  the  sum  of  the  absolute  values  of  all  kernel  func¬ 
tions  is  not  infmite),  the  nonlinear  response  can  be  expressed  by 
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x{t)  =  ho+  hi{r)^{i  c)ctr  +  /!2(ri,r2)^(r-Ti)^(r-T2)<*'i*2 


00  00  00 

/// 


^00  -  00  —  00 


hitu  T2T3)?(r  -  Ti  )^(r  -  r2)C(r  -  Ti)dt\di2dfi 


00  00  00 

—  W  —00  —00 


/i/(ti,T2  .  .  .  r,)  ^{/-ri)^(r-r2)  .  •  .  ii{t-x,)dtidxz  .  .  .  dz, 


OD  OD  OO 


—  00  —  00  -  00 


r„)?(r-Ti)$(/-T2)  .  .  .  C(r-T„) 


dtidiz  ■  .  .  (h„,  (9.27) 

instead  of  by  Eq.  9.16  for  the  linear  case  as. 


x{t)  =  ho  + 


I 

—  00 


h(T)C(t-T)iiT. 


(9.28) 


Here  in  Eq.  927 


h„(Ti,T2  .  .  .  r„)  = 


Hrk(0\,(02  .  .  .  Q)„) 


X  exp(/a>iTi +  i<W2^'2+  •  •  •  +  i(o„Tn)da)idoi}2  .  .  . 


n  =  0  00. 


(9.29) 
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■  T„)  is  called  the  n  th  order  impulse  response  function  and 

Hn((i)\,u}2  .  ■  .  (On)  the  /?  th  order  frequency  response  function.  The  expansion  of  x(r)  in 
Eq.  9.29  is  called  the  functional  expression  or  the  Voltera-Wiener  expression.  The  treat¬ 
ment  of  nonlinear  processes  from  this  stand  point  and  the  application  of  polyspectra  in 
the  analysis  of  the  nonlinear  response  system  is  dealt  with  in  Chapter  1 1 . 

When  the  nonlinearities  are  expressed  explicitly  in  the  form  of  the  equation  of 
motion  as 


Mx  +  B{x)  +  K{x)  =  F(t), 

with  the  nonlinearity  in  the  damping  and  restoring  terms,  there  have  been  a  few  efforts  to 
overcome  the  difficulties.  In  Chapter  10,  these  efforts  will  be  summarized. 

Hasselman^'  formulated  the  equations  of  motion  for  a  quadratic  nonlinear  ship’s 
motion,  expressing  the  wave  field  by  surface  displacement  •*.  f)  =  and  its  normal 
surface  velocity  (d^/dr)  (x,r)  s  ^2-  where  x  -  (xi.xi)  is  the  horizontal  Cartesian  coordi¬ 
nate  vector.  He  derived  a  generalized  nonlinear  equation  expressed  in  functional  form 
and  made  it  clear  that,  through  cross-spectrum  and  cross-bispcctrum  analysis,  it  is  possi¬ 
ble  to  get  the  linear  and  quadratic  frequency  responses  from  the  data  obtained  from 
irregular  waves  on  the  ocean.  For  his  formulation,  the  frequency  components  that  are  the 
sum  and  difference  of  two  component  frequencies  appear  to  be  important.  His  expression 
is,  however,  in  generalized  form  and  is  not  necessarily  adequate  for  practical  applications. 
Almost  the  same  content  will  be  explained  later  in  scalar  form  in  Section  11.3,  for  a  gen¬ 
eral  dynamic  system  with  quadratic  nonlinear  characteristics. 
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CHAPTER  10 

APPROXIMATION  METHODS  FOR  THE  ANALYSIS  OF  NONLINEAR 
SYSTEM  IN  RANDOM  EXCITEMENT 

10.1  INTRODUCTION 

Nonlinear  vibration  systems  have  been  studied  for  a  long  time.  The  equivalent  linea¬ 
rization  is  the  most  popular  method  for  handling  the  weakly  nonlinear  system,  and  the 
perturbation  method  has  been  used  to  obtain  the  equivalent  linear  expressions.  However, 
most  of  the  studies  have  been  concerned  with  systems  under  deterministic  excitation.  In 
this  chapter,  systems  under  random  or  stochastic  excitation  are  treated.  The  equivalent 
linearization  method  and  the  perturbation  method  are  treated  independently  for 
convenience  of  explanation. 

10.2  EQUIVALENT  LINEARIZAnON  METHOD 

Suppose  there  is  a  weakly  damped,  slightly  nonlinear  oscillation  expressed  as 

x  +  ax-\-p\x\x  +  oi}pc-\-h^  -  F(t).  (10.1) 

When,  for  example  in  case  of  rolling  with  moderate  amplitude,  the  damping  term 
includes  the  quadratic  term  for  viscous  damping  in  addition  to  the  linear  damping 
ax,  the  restoring  term  includes  the  cubic  term  Jcx^  that  comes  from  the  shape  of  the  right¬ 
ing  arm  curve  (stability  curve).  Generally  when  it^O,  this  system  is  called  a  hard  or  soft 
spring  oscillation  system. 

In  the  deterministic  case,  or  when  this  system  is  exposed  to  a  harmonic  excitation 
F(r),  this  nonlinear  damping  can  be  linearized  with  the  equivalent  linear  damping 


Ce 


a  +  a/  =a  + 


(10.2) 


using  the  amplitude  of  oscillation  xo. 

This  relation  was  derived  by  Jacobson  (1930),^^  equating  the  energy  dissipation  by 
the  damping  termed;  -t-/9lxbc  and  bya«Jc  during  one  period  of  oscillation  at  amplimde  xq. 
In  the  same  way,  equating  the  work  done  by  the  restoring  term  at  amplimde  xq,  the  equiv¬ 
alent  linear  restoring  coefficient  tu«^  is  as  shown  in  Fig.  10.1, 

=  +  (10-3) 
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Fig.  10.1.  Restoring  coefficient. 


However,  for  the  stochastic  case,  when  the  exciting  term  is  stochastic  in  character, 
we  cannot  use  Eqs.  10.3  and  10.2  direcUy.  T.  K.  Caughey’^  formulated  this  case  as 
follows.  Nonlinear  oscillation  can  be  expressed  as 


x  +  ax  +  (OqX  + 1]  •  g{x,x,  t) 


(10.4) 


where  rj  is  small,  g(x,x,  t)  includes  all  nonlinear  effects,  and  fit)  is  a  random  excitation. 
When  the  equivalent  linearized  damping  coefficient  and  the  lineanzed  restoring 

coefficient  a;  ig  are  obtained. 


x  +  a^'¥wltfX=fit), 


(10.5) 


and  the  error  e(x,i,  f)  is  expressed  by 


e{X,X,  t)  =  {a-aeq)x  +  {Q>l-(Oeq)X  +  TJ .  g{x,x,  t).  (10.6) 

Namely,  a*,,  o)lq  are  to  be  set  to  minimize  the  time  average 


i 

le^l=  lint  \eHx,x,t)dt. 
/  -►  00  2i  J 


(10.7) 
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Therefore  from 


ni  2| 

— ^  =  0,  2[(a  -  aeq)jp-  +  (g)o  -  o)lq)xx  +  rjxgix,  X,  r)]^  =  0  (10.8) 

dUtq 


and 


aie^l 


dm 


=  0,  2[{a  -  aeq)x,  x  +  icoQ-  (07q)x^  -  rixgix,  X,  t)f  =  0.  (1 0.9) 


eq 


If  this  process  Jt(f)  is  stationary. 


[xi:]  =  0.  (10.10) 

Therefore  from  Eqs.  10.8  and  10.9 

aeq  =  a  +  T}[xgix,x,  t)]l[x^],  (10.11) 

(»lq  =  (10.12) 

If  we  assume  ergodicity,  the  time  average  can  be  replaced  with  the  ensemble  aver¬ 
age,  and  therefore 

aeq  =  a  +  ii .  E[xg{x,x)yE[x^],  (10.11’) 

(olq  =  o)l  +  ri.  E{x,gix,x)yE[jp-].  (10.12’) 


This  shows  how  a*,,  <u*^  can  be  estimated  from  the  original  expression  Eq.  10.4.  When 

the  form  of  17  •  g[x,x,  t]  is  given,  the  equivalent  linearized  values  o)\q  can  easily  be 
obtained  by  Eqs.  10.11’  and  10.12’. 

For  example,  for  a  Duffmg  type  oscillation, 

+  +  =yvf),  (10.13) 
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^[j:)  -j^,  t]  =  ewo-  When  e  >  0,  we  call  it  a  hard  spring  system,  and  when  e  <  0,  a  soft 
spring  system. 

From  Eq.  10.12’ 


=  +  (10.14) 

When  nonlinearity  is  weak,  x  can  be  approximated  as  Gaussian,  and  then 

E[x^\  -  3(£[x2])^  (10.15) 

and  thus  =  Wo[l  +  3ecri].  (10.16) 

When  every  coefficient  is  linearized,  the  response  can  be  obtained.  If)t0  is  almost 
white  in  the  important  range,  £[r^]  for  this  case  can  be  computed  as 

»  al-3€a^.  (10.17) 

Here  is  the  variance  for  linear  oscillation.  Eq.  10.17  shows  that,  whene  >  0,  E[x^]  is 
smaller  than  by  nearly  3  . 

Applying  these  general  results  (Eqs.  10.11  and  10.12),  L.  A.  Vassilopoulos’^ 
obtained  andoii^  for  ship’s  rolling,  expressed  as 

jc  +  ca+^lxli  +  cuo  ^  +  =/(r),  (10.18) 

as 

Qeq  =  a  +a«  =  a  +  J%/7t  P  Oi  (10.19) 

=  <Uo  +  Poi]  k.  (10.20) 

Further,  assuming  that  the  damping  is  small  and  the  spectrum  of  the  wave  slopes  is 
flat  in  the  important  range  for  rolling,  i.e.,  (OoOx ,  he  derived 
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a+at  =  a  +  J^/n-^  ■(0(Px- 


(10.21) 


Comparing  this  expression  with  Eq.  10.2  for  the  deterministic  case  gives 


1.6 

0.85 


Ox  *  2  Ox, 


(10.22) 


namely  for  our  interest,  xq  appeared  almost  equal  to  the  significant  amplimde  for  the 
stochastic  case. 

P.  Kaplan^^  used  this  general  result  and  calculated  the  ship's  rolling  for  the  same 
example  that  this  author  solved  by  the  perturbation  method  as  shown  in  the  next  section. 
Kaplan  found  that  the  result  checked  very  well  with  this  author’s  result. 

10.3  PERTURBATION  METHOD 
103.1  Trial  for  Ship’s  Rolling 

This  author  showed^^  the  results  of  the  computation  of  nonlinear  rolling  using  the 
perturbation  method.  The  effect  of  nonlinear  damping  was  investigated  first.  One  degree 
of  freedom  rolling  is  expressed  as 


±  N2(<p9  +  Ki<f>=M,  (10.23) 


^  +  w^p  =  miO  ■  (10.24) 

Starting  with  a  0  order  approximation  (when  =  0),  that  is  linear. 


^’o  +  2a0o  +  o}^o  =  m(t) , 


(1035) 


QO 


00=  m(t)dt 


(t)  m(t-x)dx. 


(10.26) 


Here  h^om(x)  is  the  impulse  response  of  the  0  order,  name!,;  approximation  to  the 

exciting  moment  m(t). 
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=  ^  I  dr.  (10.27) 

~ao 

Frequency  response  is  easily  obtained  by  Eq.  10.25. 

Then,  when  the  nonlinear  damping  term  is  shifted  to  the  right-hand  side  of  Eq. 
10.24  and  <po  is  used  in  the  nonlinear  term,  the  first  approximation  is  expressed  by 

<pi  +  2cuf>i+a}^i  =  (10.28) 

As  the  left-hand  side  of  Eq.  10.28  is  in  the  same  form  with  that  of  Eq.  10.25,  using 
h^{x)  as  the  impulse  response  function  gives 

00 

^1=1  (10.29) 

-OD 

s  <po(t)-<f>[  (t).  (10.29’) 

Since  <7  and)3  are  small,  the  convergence  of  this  approximation  is  assumed.  Then,  the 
correlation  function  is 

/?0,^,(r)  =  £[{^o(f+r)-^i  (^+7)}(^o(0-^I  (0|*] 

(*  Shows  the  conjugate) 

=  £[0o(f  +  x)cf>o*m  -  2  {£[0i(r  +  T)0i(O] )  +  E[(f>{{t  +  r)0l*(f)] . 

(10.30) 

here  SRc  { • }  indicates  to  take  the  real  part  of  a  function  { •  ) . 

The  spectrum  is  manipulated  by  taking  the  Fourier  transformation  of  /{:^,^,(t), 
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S^,^,(£0)  =  S^^„(£0) -  2  •  “>  ■  Im^H^^iw) 


,|2  A ^2,. .2 


+^^l//^^(cwr  —a^w%^,{(jD)  +  - — 2"  S^^,{a))  ,(10.31) 

•  3C 


where  /m  { • }  indicates  the  imaginary  part  of  a  function  1  •  } . 


00  00 

5(^000  I  ^  S^Oii>o((^-(^i~f^2)da>ida)2.  (10.32) 


In  the  computation  of  these  values,  the  expected  values  £[|  •  j?  •  b/l]  and  £[|,  1^1  • •  Ij/I]  , 
that  include  the  absolute  values  l|l,  byl  must  be  calculated.  Here  i  and  rj  follow  the  two 
variables’  Gaussian  distribution. 


P(i^r])  = 


2jt0^tj}ll -Q^ 


2oy,(i-eb 


(10.33) 


(e  is  the  correlation). 

The  expected  values  of  the  two  variables  such  as  £[^,»7^]  £[^^  rj^]  have  been  computed 
by  Isserlis,’®  a  long  time  ago,  but  no  reference  was  found  on  the  expected  values  of  the 
products  of  variance  that  include  the  absolute  values  as  £[^,17,  l>7l],£[^  •  1^1  •  17  •  I17I] . 
These  were  calculated  by  this  author  and  were  found  as 


£[^-J7  b7l]  =  y^  Q-a^, 


(10.34) 


2o^  I - 

£[|.|||j7-b7l]  =  -^  yr^-3e  +  2(l  + 

7t 


yi+e  +  yi-e 

Ik  J  m 


(10.35) 


A  numerical  example  for  a  ship  model,  with  a>o  =  3.85,  (a/a>o)  =  0.06705. 

P  =  0.08,  and  waves  with  a  Neumann  type  spectrum  and  a  peak  ata>o.  is  shown  in  Figs. 
10.2  -  10.5.  Figure  10.5  shows  the  overall  effect  of  nonlinear  damping,  and  Fig.  10.4 
shows  the  effects  of  nonlinearity  on  the  term  of  Eq.  10.31,  especially  the  double  convolu¬ 
tion  (ty)  of  the  spectrum  shown  by  Eq.  10.32. 

In  the  same  way,  the  author  also  calculated  the  effect  of  a  nonlinear  restoring  force 
expressed  as 


/^'  +  N(p  +  (Klip  +  K^ip^)  =  M , 

(10.36) 

0  +  2a^  -t-  to^  +  kyip^  =  m(t) . 

(10.37) 

Starting  with  a  0  approximation  for  ~  0,  the  spectrum  of  the  fust  order 
approximation^]  when  0,  is 


(10.38) 


Here 


09  09 

(")  =  I  I  -  W  1  -  (02)dO}  ido)2 .  (1 0.39) 


1032  General  Formulation  of  the  Perturbation  M ethod 

S.  H.  Crandall^^  formulated  a  general  approach  to  the  case  with  a  nonlinear 
restoring  term  by  the  perturbation  method,  as  follows. 

The  equation  of  motion  is  expressed  as 


X-k2aX  +  (ollX  +  €giX)]  =  fit)  (1 0.40) 


where  e  has  a  small  value.  Then  X  can  be  expanded  in  terms  of  powers  of  6 


(A(cd)1 


(03S) 


03S)  jOVU 


(03S)/j0Vb 
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RAdVseC^  RAD*/sEC>  rad^/sec 


15 


Rg.  10.5.  Computed  nonlinear  spectnjm  of  rolling. 
(From  Yamanouchi?^) 
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X  —  Xq  +  €Xi  +  €^X2 'i'  .  ■  • 


(10.41) 


Substituting  Eq.  10.41  in  Eq.  10.40,  gives 

[Xo  +  2aXo  +  ~./(0]  +  f  [Xi  +  2aX]  +  to5^i  +  ti>55(.Xo)] 

+  €%  +  2£tX2  +  a)^2  +  Wog(Xi)]+  .  .  .  =0.  (10.42) 

This  equation  is  independent  of  e,  so  each  tenn  with  powers  of  e  should  independently 
be  equal  to  zero. 


Xq  +  2cxXo  +  co^o  =  At) 

.Yl  +  2£[Xi  +  Q}qX\  =  —(O^iXo))  (10.43) 

X2  +  2aX2  +  <olX2  =  -<olg(Xi)  =  -m^ig’iXo). 

Usually  the  first  approximation  X  *  Xq  +  €X\  is  used.  From  Eq.  9.16 
00 

Xo  =  J  h{x)At~t)dx.  Here  Xo  is  the  linear  response.  Accordingly,  from  the  first  equation 
—  00 

of  Eq.  10.43 


^-OT 


/l(T)  = 


J  (ail- ah 


sin^ 


(10.44) 


Then  from  the  second  equation  of  Eq.  10.43 


00 

-Oil  j 


Xi(t)  =  -ail  h{x)gX(0-x)dt , 


(10.45) 


which  is  the  expression  for  the  first  order  approximation. 
Here  some  expectations  are 
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Xi  =  + 


E[X\\  =  E[Xl\^2eE[XM 


00  00 

£[X^=  j  j  Mti)  Kx2)E^t-Xx)f{t-r2)\dXi  di2 


(10.46) 


When /(f)  is  stationary. 


R/.x)  =  E\j{t)  /If  +  T)]. 


For  example,  for  Duffing  type  oscillations  as 


X+2aX  +  a)^[X  +  eX^]  =  fit). 


(10.47) 


in  Eq.  10.40, 


giX)  =  X\ 


(10.48) 


Therefore  from  Eq.  10.45, 


00 

=  -0)1  j 


ElXoXi]  =  -Q)t  h{x)E[Xdtmit-r)]dt 


00  00  OO  00  00 

=  -<Uo  I  h(j)dx  I  h(,ti)dti  j  h(X2)dx2  |  h{Xi)dt^  |  hix^)dx4 


^00 


X  E[fit -XiW-x- X2W - T - xi)J{t -x- T4)] . 


WhenZ(f)  is  a  zero  mean  stationary  random  process,  the  following  relations  apply: 
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E[Z{ti)]  =  0 ,  £[Z(ri)2(r?)]  =  R2{t2  -h), 
E[Z(ti)Z(t2)ZU2)]  =  0, 

£[Z(ri)Z(/2)Z(r3)Z(r4)]  =  /?2Ui  -  t2)E2(t3  ~  U) 


(10.49) 


+  /?2Ul  —  t-i)R2{t2  —  U)  +  ~  f4)£2(^2  ~  • 


From  these  relations  and  the  assumption  that  the  external  force  is  white  in  the  important 
range  of  frequency  in  which  the  response  shows  a  peak. 


R/t)  =  iwod(r) 


(d(T)  is  Dirac’s  function.)  Using  these  relations  in  the  second  equation  of  Eq,  10.46  gives 


E[X]]  «  al-3€ai. 


(10.50) 


It  is  interesting  that  the  same  result  was  derived  by  the  equivalent  lineariaation  method, 
Eq.  10.17. 


CHAPTER  11 


VOLTERA  EXPANSION  AND  APPLICATION  OF  POLYSPECTRA 


12.1  VOLTERA-WIENER  EXPRESSION 


Suppose  there  is  a  weakly  nonlinear  response  process  T(f)  which  is  an  output  of  a 
linear  input  process  X{t).  Both  y(f)  and  X(f)  are  stationary  up  to  an  appropriate  order. 
Then  when  the  sum  of  the  absolute  values  of  all  kernel  functions  is  not  infinite,  Yit)  can 
be  expressed  as 


Y{t) 


00 

=  1 


h{r)  X(t-t)  dT  + 


00  00 

II 


Xit-Xi)  X{t-X2)  dxidxi 


00 

III 


h?,(ri,X2,Xi,)  X(t-xi)  X(r-X2)  X(t-x^)  dxidx2  dx^  -  ■  ■  (11.1) 


~Q0 

00  00 


=  X  I  I  ■  ■  ■  /  •  -O  X{t-Xi)  Xit-X2)-  •  -Xit-Xn) 


•^CO  .^00  ^00 


cix^dx2  *  *  ■  dXf^ 


(ii.r) 


Here  h„(x\X2  •  •  •  t„)  is  a  real  function  of  n  variables  -  »  <  r,  <  +  oe ,  i  ®  1, 2,  •  •  •  n 
and  is  called  nth  kernel  function.  All  kernels  are  assumed  to  be  smooth,  absolutely  inte- 
gratable,  and  to  possess  Fourier  transforms.  Besides,  all  these  are  supposed  to  be  time 
invariant.  When,  for  any  r,  <  0,  h„(,Xi,X2,  •  •  •  t„)  =  0,  then  the  lower  limit  of  these  inte¬ 
grals  can  be  0.  For  n  =  0,  Ao  is  the  value  of  Yit)  when  X(f)  =  0.  So  we  can  include  n  =  0, 
generally;  otherwise  we  assume  ho  =  0.  When  n  =  1,  Yit)  will  be  the  linear  system  that 
has  been  treated  in  Part  I  and  II  and  as  Eq.  9.16 

00 

Yit)  =  f  hix)  Xit-x)  dc.  (11.2) 


Equation  11.1  is  assumed  to  be  a  kind  of  Taylor  expansion  of  Yit)  around  the  linear 
proce':s  expressed  by  Eq.  11.2.  Accordingly,  the  terms  for  n  S  2  are  regarded  as  modify¬ 
ing  SF  dll  terms. 

If  we  change  the  variables. 


Yit) 


00  00  eo 

-ill  - 1 


h„it-Xx,t-X2,- 


■t-Xn)XiXx)  XiX2)-  ■  XiXn) 


dt\dx2  -  •  ■  dt„  .  (11.3) 


Equation  11.1  or  11.3  is  called  the  Voltera-Wiener  or  functional  expression. 
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1 1 .2  fflGHER  ORDER  RESPONSE  FUNCTION. 

hn{r\,X2,-  ■  •  -tOn) 

When  h„{x\,X2,  ■  •  •  t„)  are  smooth  and  absolutely  integratable,  the  nth  order  Fourier 
pair  exists  as  was  stated  at  the  beginning. 


An(Tl,T2,- 


OB 

/ 


exp/|<uri +02  f2  +  '  •  ■+<yn  Tn}  H„(U)uU}2,  ■  '  ■  CD„} 

d(Oi  aa>2  •  •  •  da}„,  (11.4) 


H„i(0u<02,- 


•(On)  = 


/ 


oo 

j  exp[-i(coi  ti  +  £U2  T2  +  -  •  ■  +  tu„r„)] 


hnituTir  ■  -tn)  dtidxi-  '  '  dt„,  (11.5) 

K(J\,X2,-  •  -Tn)  are  real  functions  and  are  symmetrical  for  ti,T2-  •  -Tb, 
thus 


hn{rx,X2,  -  ■  •Tn)  =  *n(r2,ri,-  •  •  r„)  =  •  •  ■  =h„(X2,ri,-  ■  ■  X„,Xi)  (11.6) 


H„(a}i,(02,-  ■  (0n)=H„(a>2,C0u-  •  (0„)=-  •  •  =  H„{(02,(0^,-  ■  ■(o„,(t)x).  (11.7) 

If  we  regard  y(f)  as  the  output  and  it(f)  as  the  input  process,  h„{x\,X2,-  •  Tnlis 

called  the  nth  order  impulse  response  function  and  //„(oi,<U2,  •  •  •  o„)  is  the  nth  order 
frequency  response  function. 

When  the  processes  are  not  continuous  but  discrete,  then  the  Voltera-Wiener 
expression  for  the  nonlinear  response  Y,  and  linear  input  X,  is 


8nv  ^l—u  ^l—v 


u  =  ov  =  o 


00  oe  X 


^  ^  Suvw  ^t-u  ^t—V  ^t—W  +  ■  •  ■ 

(11.8) 

u  =  ov  =  o<o  =  o 

00 

=x 

(11.8’) 

n=l 

n 

and  the  following  discrete  Fourier  pair  is  assumed  to  exist: 


296 


I 

I 


Gto,,a<2,.  .  •<».)=  X  1  ■  ■  1  gu.u,--  u. 


2**‘'  ■ 


Ui  s  o  U2  ^  O  =  ^ 

;r  ;r 


(11.9) 


d(jD\d(02 . (11.10) 


For  convenience  the  expressions  for  a  continuous  process  will  be  used  in  this 
chapter  unless  otherwise  stated. 

11.3  SECOND  ORDER  NONLINEAR  PROCESS 
BISPECTRUM,  CROSS  BISPECTRUM 

For  example  in  Eq.  11.1,  the  quadratic  nonlinear  characteristics  of  Y{i)  is  expressed 
by  /j  =  2,  and  the  terms  n  >  3  can  be  neglected.  Then  Eq.  11.1  becomes 


Y{t) 


=  1 


hi{t)X(t-T)dz  + 


h2{x\,X2)  X{t-xi)  X(r-X2)  dxidx2,  (H.H) 


which  corresponds  to  Eq.  11.2  for  a  linear  process.  As  the  expected  values  of  the  second 
moment,  the  auto  or  cross  covariance  functions  Rxxix),  /?n<r),  and  /irx(x),  and  their 
Fourier  transforms,  the  auto  or  cross  spectra  sxx(o>),  syrico),  and  syxico)  played  big  roles 
in  the  analysis  of  the  responses  of  linear  systems,  the  expected  values  of  the  third 
moments /?xxx(ti,T2),  ftyyy(Xi,X2),  /?Kcr(Ti»T2).  and  their  double  Fourier  transforms 

jXxx(<wi,<02),  are  important  in  the  analysis  of  second  order 

nonlinear  processes.  The  third  moment  correlation  functions  are,  for  example, 

^m<Xi,r2)  =  E[ly(r  +  Ti)-mr}ly(r  +  X2)-fnr}ly(t)-mr}  (11.12) 

Exxy(xi,X2)  =  E[X(t  +  Xi)  X(r  +  T2)(T(t)-"Jy}].  (11.13) 

Here  £[X(r)]  =  0  is  assumed.  Their  double  Fourier  transforms  are  called  the  bispectrum 
smiof  1,(02)  or  the  cross  bispectrum  sxxy((o  1,(02), 

00  00 

sm<(0i,(02)  =  j  j  Eyyy(XuT2)cxp[- i((0ixi  +  m2T2)]£#rit#r2  (11.14) 


.00  ^00 
00  00 


sxxy((0i,(02)  =  -^^  j  j  ExXY(Xl,t2)cxp[- i((0 1X1+ (O2T2)]  dXidX2.  (11.15) 


The  inverse  transforms  are 
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00  00 


5yy}<a>  1, 0)2) exp[j(WiTi +  0)2^2)]  d(JD\d(02  (11.16) 


.00  —00 
00  00 


/?yyy(Ti 


■SXX}<i£>l.<y2)exp[j(fl)iTi +  0)2T2)]  d(i)id(i)2-  (11.17) 


When  the  input  process  X{t)  is  a  Gaussian  process,  then 

^xxx(^^i.T2)  =  £[X(r  +  Ti)  X(f +  T2)  X(r)]  =  0.  (11.18) 

Therefore 

SXXXi(0\,(i)2)  =  0,  (11.19) 

The  bispectrum  does  not  exist  for  a  linear  Gaussian  process. 

For  a  quadratic  (second  order  nonlinear)  process  y(r),  RyTrituTz)  *  0, 

^xxy(Ti.T2)  ^  0  and  the  bispectrum  exists  as  shown  by  Eq.  11.14  and  Eq.  11.15. 

Now,  assuming  Y{t)  is  a  stationary  process  and  its  Fourier-Sheltjes  integral  is 


[Yit)-mY\=  I 


dZiw)txp(Jo}  t). 


(11.20) 


then 


00 

(r(f +  Ti)- my}  =  I  r£Z(G)i)exp(/a)ir+ ioi.Ti), 


^00 

00 


(y(f  +  T2)  -  my)  =  J  dZ(af2)  exp(i(02r  +  10)2X2)- 


Therefore 


RYYY{x\,r2)  =  £[((X(r  +  Ti)  -  my|fy(r  +  X2) -  my}{y(r)  -  my}] 


=  /// 


exp[j((o  1  +  0)2  +  W3)f]  •  exp[i(a)  \Xi  +  0)2X2) 


E[dZ{o)i)dZ(o)2)dzio)i)\  (11.21) 

[60  is  expressed  as  0)3]. 

Because  of  its  stationarity,  .^yry(ri,r2)  is  a  function  only  of  ti,  r2 ;  it  is  not  a  func¬ 
tion  of  t  but  is  independent  of  r.  Accordingly,  the  integral  has  a  value  only  when 

0)1 -1-0)2 +  0^3  =  0  or  <W3  =-Wl-tt>2  (11.22) 
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comparing  Eq.  11.21  with  Eq.  11.16,  and  taking  into  account  the  relation 

J_ 

2jt 


W 

I  exip[i(a)i+a)2-^o)^)tdt  =  d(a)i'¥W2^o)^)], 
271  J 


00 

I 


£[dZ(a>i)  dZ{o)2)  dZifoi)]  6{o)\-^(i)2-^oii^)dwi~SYYyio>\,(t>2) 

-®  d(D\d(i)2-  (11.23) 

This  shows  that  £[dZ(0])  dZ(a>2)  d2{(Oi)]  must  be  0  except  along  the  line 

(0\+<02  +  <W3  =  0.  This  corresponds  to  the  fact  that,  for  the  linear  case, 

00  00 

£xx(r)  =  11  exp  i(a)+Q)')t  exp  itoT  ■  E[dZ(a) )  •  dZxo '  ] 

~00  ~00 
00 

=  [  exp/ru  T  S2odo>)  dw. 


(11.24) 


00 

and  j  E[dZ((i))dZ(a)')]d(a)+(o')do)'  =  sxx(fl>)di(o). 

—  00 

By  the  same  logic  that  because  of  stationarity,  /?xx(t’)  should  be  independent  of  t,  there¬ 
fore  E[dZ(Q))  dZ((o')]  must  be  zero  except  whency  +  a>'  =  0,  i.e.,  only  along  the  line 
0)  +0)'  =  0  ando;'  =  -<u. 

Because  of  the  symmetry  of 

£xXX(ri,T2)  =  £jCKX<72,7i)  = /?X3CX(-ri,T2 -Ti)  =  £xxy(^'2 -T'l.-ri) 

=  £xXX(-T2.Ti-T2)  =i?xXX(^'l-T2,-r2)  (11.25) 

the  spectra  also  have  symmetry, 

Sxxx{<0u(02)  =  Sxxxi<‘>2^<0l)  =  SxXXi(0u-(0 1-0)2)  =  SXXX(.-0)1  -0)2,0)i) 

=  SxXXi<02^-<0l-<02)  =  SxXXi-0}\-0)2,0)2)  (11.26) 

and  sxxx(o>ifO>2)  =  s*  (-a)i,-Q)2).  (11.27) 

Accordingly,  when  we  think  of  the  bispectrum  sxxx(0)i,0)2)  we  must  consider  the 
third  frequency  a>3,  besides  Oi  andu;2<  that  makes 

(01+0)2  +  0)3  =  0,  a)3  =  ~0)i-0)2- 

From  Eq.  11.26,  the  bispectrum  is  found^®  to  be  the  same  for  six  permutations  for  two  of 
the  three  frequencies  o)  1,(02,  and  0)3.  Twelve  bispectra,  including  the  conjugate  bispectra 


i 

i 
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show  the  same  modulus  as  is  shown  in  Fig.  11.1  for(^  to(^.  These  bispectra  are  repre¬ 
sented  by  a  bispectrum  in  the  first  octant  where  j£U3:  >  W]  ^  a>2.  The  bispectrum  is 
shown  by  the  modulus  and  the  phase  angle. 


At 


Rg.  11,1.  Symmetric  character  of  bispectrum. 

(From  Yamanout^i  and  Ohtsu.^®) 

The  points  marked  O  in  sub-octants  and  show  the  same 

bispectra,  equal  to  sxxx(ofu<02)  at  mark  O  in  octant  and  the  bispectra  at  points  x  in 
the  six  sub-octants  (s),  (^,  and  (j^  show  its  conjugate.  Also  in  Fig. 

11.1,  the  segments  of  line  in  each  sub-octant  are  shown  marked  by  short  lines  ■ft',  -fif 
that  correspond  to  the  segmented  line  AB  4-,  BC  -fr,  CD  -Hf  in  octant  that  shows  the 
same  bispectrum.  At  a  fixed  frequency  in  Fig.  11.1,  the  bispectrum  along  the  segment 
BA  (cob  =  tt/3)  shows  interference  with  the  two  components  at  the  smaller  frequencies 
a>  1  and  a>2,  that  makes  o)  j  +  (U2  =  a>£  =  m3;  the  specmun  along  the  segment 
BC  ((Ob  =  a>i)  shows  interference  with  one  smaller  frequency  component  a>2  and  one 
higher  frequency  component  <U3(m3  =  (O2  +  «i);  and  the  bispectrum  along  the  segment 
CD  (cob  =  (02)  shows  interference  with  the  components  at  two  higher  frequencies 
a>3(a>3  =  0)1  +  0)2)  and  0)1.  The  hexagonal  boundary  shape  in  Fig.  11.1  shows  the 
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boundary  of  the  frequency  region  in  which  the  bispectnun  can  be  calculated  for  the  data 
sampled  by  interval  At . 

Figure  11.2  is  an  example  of  the  bispectrum  of  the  rolling  of  an  actual  ship  on  the 
sea,  reported  by  Y.  Yamanouchi  and  K.  Ohtsu.^**’’  The  bispectrum  in  Fig.  11 .2  shows 
higher  values  along  a  similarly  shaped  segment  ABCD  in  Fig.  11.1  with  tug  at  the  fre¬ 
quency  at  which  the  linear  spectrum,  drawn  down  the  side  of  the  a>]  axis,  shows  the  peak 
value,  that  implies  the  nonlinear  interference  is  higher  along  these  shaped  segment  lines, 
as  ABCD. 


EXAMPLE:  45’  -  0.45  X  10’  RAD 
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0.5 

ROLLING  BISPECTRUM 
SEIUN  MARU 


1.0 


COi  (1/SEC) 


(ARGUMENT) 

ROLLING  B.'SPECTRUM  SEIUN  MARU  (Argument) 

Hg.  11.2.  An  example  of  bispectrum  of  rolling  of  a  ship  on  the  sea. 
(From  Yamanouchi  and  Ohtsu,^®'^®  (1972).) 
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As  has  been  formulated  in  Eqs.  11.12  and  11.6 


£[{y(r)  -  my}{y(r  +  Ti)  -  my}{y(ri  +  T2)  -  my)  =  /?yyy(Ti,  T2) 


00  00 

=// 


5(<yi,t02)exp[+  J(a>iTi  +  (O2T2)]  d(x)\da)2 


thus 


00  00 

£|(y(r)-Wy)^  =/?yyy<0,0)=  j  j  S(a)i,(02)ck0ida)2-  (11.28) 


The  bispectnun  shows  the  contribution  of  the  third  moment  of  frequency  components  at 
three  frequencies  a>  1 ,  a>2,  and  <03,  where  <y  i  +  a>2  +  m3  =  0  are  satisfied.  When  square 
mean,  cubic  mean,  and  fourth  power  mean  are  expressed  as 


E  (y(r)  -  my]^  =  a  ^ .  £[{y(f)  -  my)^ 


the  statistical  values  called  skewness  s,  and  peakedness  p  are  defined  as 


If 

(11.29) 

__  t*'' 

(11.30) 

When  y(f)  is  a  Gaussian  (not  nonlinear)  process,  =  0,  p*  =  30* .  Therefore, 

s  =  0,  p  =  3.  By  the  values  of  s  and  p,  we  can  show  the  extent  of  nonlinearity  of  the 
process.^’ 


11.4  CHARACTERS  OF  QUADRATIC  RESPONSE  TO 
GAUSSIAN  INPUT  PROCESS 

As  already  was  shown  in  Eq.  11.2,  whenX(r),  the  input  process  is  a  Gaussian  linear 
process  with  £[X(f)]  *  0, 

Rxxxkci,xz)  =  £[(X(f +  Ti)  X(r  +  T2)  X(f)]  =  0  (11.31) 

S2Q0(i<0\,(02)  =  0.  (11.32) 

Several  statistical  relations  of  the  quadratic  nonlinear  output  Y{t)  to  input  X(r)  and  their 
derivations  will  be  summarized  as  follows: 

1.  Mean  of  y(r),  £[y(f)]  s  my. 
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From  Eq.  11.11,  and  as  £[X(/-t)]  =  0, 


my  s  £[y(r)]  = 


00  00 


E[X(r-Ti)-X(t-T2)]  dr  diz 


>.00  >.00 


00  00 

=11 


hiJuTi)  Rxxiti-ri)  dxxdxi 


(11.33) 


h2itx,X2)- 


j  i'x^cu)exp/w(r2-Ti)  do) 


^  dtidti 


00  00 

=11 


h2{x\,X2)^r.'pi{o}X2-(or\)  ctt\dr2  sxx(0)  da) 


.00 

1 


OD 

=  1 


H2ia},-a})sxx  Uo)  do) 


(11.33’) 


because 


^2(^1.  T2)  exp  i{a)X2  -  Q)xi)dxi,  dx2-  H2{o>,-(o) 


—  00  —00 


2.  Cross  correlation  £ix(t);  cross  spectrum  syx(tt>)- 
For  /?ix(r)  =  E[[Y(t)  -  my}  •  X{t  -  r)]  =  £{7(0  •  X{t  -  r)}  -  my  E[X{t  -  r)], 
inserting  Eq.  11.11  and  £[X(f-r)]  =  0,  yields 


00 

=  I  hi{Xi)E\Xi 

xOO 


t~Xi)X{t-x 


00 

)]^1=  j 


(11.34) 


Therefore 
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00  00  oc 

5yx(ty)  “  j  ~  ^  {  I  e~‘^dxidx 

—  00  —  00  —  00 


J^ 

2ji 


I 


—  00 


/ii(r)  ■  e‘*^'dt 


i  • 


—  00 

I  /?(T-Ti) 

OP 


=  //l(Cj)5X?f(^0) 


(11.35) 


and  Hii(o)=  SYx((o)/sxxi(o). 


(11.36) 


Eq.  1 1 .36  shows  even  when  y(r)  includes  the  nonlinear  component,  if  the  input  A((f ) 
was  a  Gaussian  process,  the  linear  response  frequency  function  H\{u))  is  expressed  by  the 
ratio  of  syx  (<o)  and  sxx  (o))  as  in  the  purely  linear  case. 


3.  Autocovariance  Ryy  (r);  linear  spectrum  syy  (w). 

As  in  the  linear  process,  the  Fourier  transfonn  of  second  order  moment  or  the  auto¬ 
covariance  function  is  called  the  linear  spectrum, 

Rryir)  =  E[Yit)  -  my][F(r  +  r)  -  my] 

=  £[7(0  •  y(r  +  T)]  -  m}.  (11.37) 

Here  by  the  use  of  the  fimctional  expression  of  Y(t)  and  y(r  +  r)  as  Eq.  11.11  and  appro¬ 
priate  variables, 


£yy(r) 


=  // 


^i(si)  /ti(ri)  E[X(f-Si)  Xir  +  T-ri)]  dsidri 


X  00  00  00 

iia 

—  X  —X  —X  —X 


*2(«1,W2)A2(Vi,V2)  £p:(f-tti)  X{t-U2)  A:(f  + T-Vi)  X{t^X-V2)\ 


duidu2dvidv2  -  ^y- 


(11.38) 


Generally,  when  Xi,  X2,  X3,  X4  are  the  probability  variables  that  are  from  a  joint  Gaussian 
distribution,  then 

E\Xi  X2  X3  X4]  =  Mi2  M\i  M24  +  M\4  Af23-  (11.39) 

Here 

Miic  =  E[XrXi,]. 

From  this  relation. 
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£[X(r-wi)  X(r-«2)  X(r  +  T-vi)  X(/-t-T-V2)] 

=  -^2)  ^JciKvi  -V2)+  +  wi -vj)  RxxfJ  +  U2-V2) 


+  ^XX(^’+“l-V2)  £xx(T  +  «2-Vi). 


(11.39’) 


When  this  expression  is  inserted  in  Eq.  11.38,  the  term  that  comes  from  the  first 
term  of  Eq.  11.39’  is  the  same  as  in  Eq.  11.33  and  it  cancels  with  the  last  term  of  Eq. 
11.38,  -OTj. 

Then  from  the  remainder 


Ryy(X) 


00  OD 

-1/ 


Mil)  M^i)  £xx(f + 


—  00  —00 
00  00  00  00 

iin 

—  00  —  00  —  00  —  00 

00  00  00  00 

//// 

—  00  —  00  —  00  —  00 


hii^uui)  h2(yi,v2)  Rxxxir ui-vi)  /?xx(t  +  «2 - ^2) 

du\du2dv\dv2 

h2(u  1 ,  U2)h2iv  1 ,  V2)/?XXX(T  +  U  1  -  V2)/?XX(T  +  U2  -  V 1 ) 


du\du2dv\dv2.  (11.40) 

The  terms  /i2(«i,  «2).  ^2(vi,  V2)  are  symmetrical  for  «i,  uz  and  vj,  V2,  and  the  scope  of  the 
integral  is  the  same  as  for  -  »  —  +  00  for  both  the  last  two  terms.  Accordingly,  the  sec¬ 
ond  and  third  terms  are  equal.  Therefore, 


SYY(a>) 


£it(t)  e  dr 


•ij 

—  00 

00  00  00 

Hi(a)j\^sxxio})^  11^1  exp[-/(<y-(<yi -)-a>2)k3  dx 


—  00  —  00  —  00 


X  2H2*  ((0\,U)2)  H2((0i,(02)  SxA(Ol)Sxx  {(O2)  d(0\d(jD2.  (11-41) 

Here,  using  the  relation 


J_ 

2ji 


00 

1 


exp  [z(((Wi  +  t02)-fl>jr] 


dx  =  d{(ty  1  -t-a>2)-w} 


(11.42) 


6:  Dirac’s  delta  function. 
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inEq.  11.41  gives 


SYYi(o)  =  \Hi{(o)rsxx{of) 


+  2  J  ^  \H2((0  1,(02)^ Sxxi^l) 

—  00  —00 

00 

=  ^liojfsxxiio)  ■*■  2  I  i^2(^y -(02,i02fsxxi(i>-(i^2)Sxx(W2)^(^2-  (H.43) 


This  shows  that  the  linear  spectrum  sy}<a>)  is  the  sum  of  the  spectrum  of  the  linear 
part  j//i(a))psxv<to)  and  the  modifying  term  in  Eq.  1 1 .43  defined  by  the  second  nonlinear 
response  function  H 2(0) -0)2, 0)3)  2nd  the  product  of  two  linear  spectra  -<^>2)  and 
sxxitoz). 


4.  Cross  correlation  of  third  moment /?jxx‘.  cross  bispectrum,  syxk  (c).  and  the 
second  order  frequency  response  function  //a  (<u  1 ,  coi) . 

The  cross  correlation  of  the  third  moment,  for  example  Ryxx  is 


^m<Ti,T2)  s  £[{y(t)-my|  X(r-Ti)  X(t-X2)] 


s  £[y(f)X(f-ri)  X(f-r2)]-my£[X(f-ri)  Xit-n)].  (11.44) 

Here  expressing  Y{t)  by  its  functional  expression  Eq.  11.11  and  using  the  same  relation  as 
Eq.  1 1 .39’  and  Eq.  1 1 .33  for  m  y  gives 


^m(7i,r2)  = 


h2(MuU2)  E[X{t-ui)  X{t-U2)  X{t-Ti)  X(,t-X2)]  duidu2 


—  OQ  —00 


-my  Rxxir  1-X2) 


00  00 

~  j  j  ^2iUi,U2)[RxxiUi-U2)  ■  Rxxiti-X2) 


+  Rxxi'^l-U  1)  Rxx(J2-U2)  +RxxiT2-Ui)  ^XX<Tl-M2)]  du\dU2 
h2(y\,  vi)  Rxx^n,  ^2)  dvidv2  Rxxk^i  -  tz).  (i  1  -45) 


// 


In  this  equation,  the  first  term  and  the  last  term  are  the  same  and  cancel  each  other.  The 
second  and  the  third  terms  are  the  same,  as  h{u\,  uz)  is  symmetrical  with  u\  and  uj,  and 
the  range  of  integrals  is  ft’om  -  «  —  +  00  for  both. 
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Therefore 


OC  00 

RYXxirhr2)  =  2  J  I  h2{ui,U2)  Rxxiri- ui)Rxx{r2- U2)  duidu2 


(11.46) 


and 


'  —00  —00 


^raK7i,T2)exp[-/(a)iTj  +a>2T'2)]  dxidT2 


2 

TTTT  ^2(Ul.l 

(27r  J 


«2)  ^xx<Ti  -  ui)Rxx{r2  -  W2)exp[-  /(win  + 1/>2^'2)] 

dx\dT2  (11-47) 

1,0)2)  =  2  H2{<JD\,0)2)  ■sxxfoi)  •  Sxxi(02)  (11.48) 

5yXX<0)i.W2) 


^2(0)  1,0)2)  = 

2sxx(<i>l)  Sxxit^z) 

Here  with  the  variables  eoi,(02  into  Wi'.wi'  inverted  by 

0)1  -  +102) 


(11.49) 


0)i-0>2  =  0/1 

0)l  +  0/2  =  0/2' 


(11.50) 


0/2=— (-0/1  +0/2), 


then 


,yrax(Q)i,  0/2)  =  ^skoKo)  1  - 0/2, 0/1  + 0/2)  =  5m(Q)  1.0/2)  J 
Since  Jacobian 


5a/ 1 

3a/2 

1 

1 

0)1,  0/2 

30/1 

3a/ 1' 

y  ’ 

~  2 

0/1, 0/2 

3a/ 1 

3a/2 

1 

1 

d(02 

d(02 

2  ’ 

I 

0)1,  0/2 
0)1  ,  0/2 


2’ 


.(11.51) 


(11.52) 


therefore 


and 


SYXsdfiii,  0)2 )  =  .ym<o)i,o/2)  x 


2  5}xiKo)i-q/2,o/i +6/2)  =  ^m<o/i,a/2). 

Substituting  into  Eq.  11.49  gives 

u  r,.  ..X  ■yKQr(o)i-o/2,o>i  +  a/2) 
^2(0)  1,0/2)  - - i ^ 

sxxifOl)  sxjdfOl) 


(11.53) 

(11.54) 

(11.55) 
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In  the  same  way,  if  we  use  X\t)  instead  of  the  output  process  Y{t)  and  calculate  the  cross 
bispectrum  S;^Jxx^£i>i,n>2) . 


=  E[X{mt)  Xit-rO  X(t-T2)]-mx2E[Xit-ri)  XCr-tj)] 
=  Rxxi^)  Rxx^T2 - Ti)  +  2/?xx(ti)  Rxxiti) -  r^x iRxxiJi  - ti) 


=  2  Rxxi'^i)  ^xxi'^i). 


(11.56) 


thus 


Sx^xxi<0 1,(02) 


00  00 


/?xx(ti)/?Ax(T2)  exp[-  /{miTi  +  m2T2)]  dridr2 


=  2  sxx((0l)sxx((02)- 


Inserting  into  Eq.  11.49  gives 

H2{(0  1,(02) 


SYXXi(0h(02) 

Sx^xxi(0h(02) 


and  inverting  the  variables  as  in  Eq.  11.55  gives 


H2{(0  1,(02) 


SYX3d0>\-<02,(0\'^<02) 
^X^XX((0 1  ~  (02>  0>  1  +  <t>2) 


(11.57) 


(11.58) 


(11.59) 


Both  Eqs.  1 1 .49  or  1 1 .55  and  Eq.  1 1 .58  or  1 1 .59  can  be  used  for  the  computation  of 
H2((0i,(02)-  However  in  actual  computation,  due  to  the  consideration  of  the  window, 

Eq.  11.58  or  11.59  is  much  better  than  Eq.  11.49  or  Eq.  11.55. 

5.  Application  of  bispectrum  analysis. 

J.  F.  Dalzell^^'  ^  applied  these  theoretical  relations  (as  compiled  above  in 

items  1  through  4)  to  the  problem  of  added  resistance  of  ships  in  waves.  The  quadratic 
process  y(t)  is  the  resistance  D(t)  of  ships  in  waves  and  the  input  process  x(t)  is  the  wave 

height  7(f)  in  Eq.  11.1’  (when  n-0  —  2)  and  ho  =  Do,  as 


OP 

/ 


D{t)  =  Z)o  +  I  Ai(7)  Ti{t-x)dx  + 


00  00 

II 


^2(^1. 72)  ri(t -Ti)  7(f-r2)  (lxidt2. 


Then  Eqs.  11.33  and  11.33’  are  the  expression  of  mean  added  resistance  in  waves 

00 

E[Dit)]=  ^  H2{(0,-(o)5^^{(o)  dm,  (11.60) 

.00 

and  the  linear  and  nonlinear  response  functions  of  resistance  to  waves  are  Eq.  11.36, 
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(11.61 


Eq.  11.55,  andEq.  11.59  as 
Hi{(0)  =  SDtii(o)/s^^ia)) 

,  SD  I  -  0)2,  (O1+W2)  So  mt  (0)1 -(02,0)1 +0)2) 

H2(0)i,(O2)  - - - - = - - - 

Sr,n(o>l)Sfjti(0)l)  S^irtr,  ((Oi- 0)2,0)  1+0)2) 

as  a  special  case  of  Eq.  11.60.  Therefore, 

„  ,  ,  ■SZ)i7i;(2h>,0) 

//2(a»,-cy)  = 12- = -r-— — . 


(11.62 


(11.63: 


Dalzell  performed  a  series  of  tests  in  a  towing  tank  on  resistance  increase  on  en¬ 
countering  head  waves,  with  sea  states  A,  B,  and  C,  where  the  significant  wave  heights 
are  in  the  ratio  of  1:2:4.  The  results  were  analyzed  and  the  applicability  of  these  theories 
was  made  clear. 

Figure  11.3  shows  the  linear  frequency  response  function  [H\(o))  =  G\(q)  in 
Dalzell’s  expression]  obtained  by  Eq.  11.61  in  real  and  imaginary  parts,  and  Fig.  11.4 
shows  the  impulse  response  function  /I](t)  that  is  the  Fourier  transform  of  H\{(o)  in  the 
discrete  form  Lit, 

/ti{(T  =  Lk  (11.64) 

Ar  being  the  sampling  time  interval. 


Fig.  11.3.  Linear  resistance  frequency  response.  (From  Dalzell. 


Fig.  11.4.  Coefficients  4  linear  resistance  impulse  response  function. 

(From  D^zell.55) 

Herea«  is  the  noodimensional  encounter  frequency  a«  =  where  a>x.  is  the 

frequency  of  the  wave  with  l^.igth  equal  to  the  model  ship  length  L,  i.e.,  where 
(Ol  =  J2ng/L .  The  added  resistance  and  wave  height  are  also  nondimensionahzed  by 
the  displacement  and  the  length  of  the  model.  Figure  11 .5  shows  the  special  case  of 
H2(.(i}[,02)  whenoi  *  (0,(02  ~-(o.  The  term  H'^(o,-(o)  is  expressed  by  Eq.  11.63,  and 
the  results  obtained  in  regular  wave  tests  are  also  shown.  The  agreement  is  excellent. 


Rg.  11.5.  Estimate  of  G2(a*,-ae)from  cross-bispectral  analysis,  Fn  =  0.15. 

(From  Dalzell.®®) 
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Figure  11.6  shows  an  example  of  the  three-dimensional  plot  of  the  modulus  of  a 
cross  bispectrum  of  wave-wave  resistance  sdi7i;(<Wi  •  (02).  Fig.  11.7  shows  the  plot  of  the 
real  part  of  the  same  cross  bispectrum. 


Fig.  11.6.  Isometric  plot  of  modulus  of  wave-wave  resistance  cross  bispectrum, 
sea  state  B,  Fn  «  0.15. 

(From  Dalzell.®®) 

The  axis  Qi.Qj  used  here  is  Q\==‘0}\-(i>2,Q2  =  o^i+<02-  Figure  11.8  shows  an 
example  of  the  real  part  of  H2(<o\,(02)  obtained  by  Eq.  11.62,  and  Fig.  11.9  shows  the 
second  order  impulse  response  function  h2itx,X2)  in  discrete  form  Qj  as 

h2{rur2)  -  Qjk  <5 (ti -JAt)  dif2-k^t)  (11-65) 

in  the  form  of  weighting  functions.  These  values  were  obtained  from  the  data  in  sea  state 
A.  Using  these  weighting  functions  Z*  (Fig.  11.4)  and  gy*  (Fig.  11.9),  obtained  from  the 
test  data  for  sea  state  A,  the  added  resistance  was  calculated  by 

m  P  P 

^(.n)=  LjTiin-j)+  Y,  ^  Qjk  (11.66) 

j  =  -m  j  =  -p  k  = 

for  sea  states  B  and  C,  where  the  discrete  readings  ijin  -  f),  Tj(n  -  k)  of  the  time  history  of 
wave  height  rj(r)  for  sea  states  B  and  C  were  used.  The  added  resistance  thus  calculated 
was  compared  with  the  real  time  history  of  added  resistance  in  sea  states  B  and  C.  The 
results  for  sea  state  C  are  shown  in  Fig.  11.10  as  an  example.  The  agreement  of  this  com¬ 
puted  time  history  with  the  actually  observed  time  history  is  surprisingly  good.  Here,  two 
digital  filters,  as  shown  in  Fig.  11.11,  were  used  to  get  rid  of  the  effect  of  the  finite  length 
of  the  digitized  impulse  response  functions. 
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Rg.  11.7.  Real  part  of  wave-wave  resistance  cross  spectrum,  sea  state  6, 
Fn  =  0.1 5  (normalization  const.  =  3.76  xicr®). 

(From  Dalzell.®®) 
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FUNCTION  IS  SYMMETRIC  ■ 
ABOUT  THE  LINE 
0,-02  : 


n  to  J*  14^4  -» 

V  4t  M  U  J  >4  -4  /s 

V»4  to  «i  to  14  la  *  -t  ^  jl 

^  tt  M  to  rr  to  *14  >14  -t 

yr,.!  4  14  21  14  '  *14  -I#  -la  •! 

^  4  4  to  14  4/ >11  •W  to  •« 

to  U  4  4  to  4  /to  o|>  >4  -1 


5  /*  i  •»*  tol  4 

/\/*»  »  « 

If"!  14  1  4 

^  to  11  to  4  2 

f  •»  44  to  14  4 

44  44  17  14 


4/  -44  -4/  4/ 

^'-41  -iv  to  /-4  •y 


^1  *  1  v->  ••V’to  I  •4,^14  4/ 

'  -  4  »  4  14  ^14  to  v«/  14  to 

^  4  10  I''  cV  «*  to  M  41  *  \  *^^ 

14  11  11  24  14  14  Ito  to  4* 

^  II  l«  2t  to  14  44  42  14  11 

/v  14  to  11  11  44  14  4 

)4  1ltoll  14to«7^-4* 
^  It  44  to  24  4  to  40  ll*“n 

44  to  14  14  to  44  41  14 

•t  44  to  22  41  24 
44  to  24  to  1C  <24 

^  24  20  /■  -1?  -XU 

^  14  14  \-4  -42  Ito 
^  21  On*  14  N 

<>  M  »  »\-|l 

>  II  It  4' 

^  >; 


REAL  PART  IS  SYMMETRIC. 
AND  IMAGINARY  PART  IS 
ANTI-SYMMETRIC  ABOUT 
THE  LINE  02-0,  ;  (Oj-O) 


\ 

\ 


Fig.  11.8.  Real  part  of  estimated  Gi(ae,-a , ).  sea  state  B,  Fn 

(From  Dalzeli.®®) 
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Fig.  11.9.  Coefficient  Qj^  of  quadratic  resistance  impulse  response  function, 
values  shown  are  50  Qjt  truncated  to  integer. 

(From  Dalzell.55) 
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OBSERVED  RESISTANCE  (NO  FILTERING) , .  POINTS  0139  THROUGH  0439 


Fig.  11.10.  Time  history  of  waves  and  resistance  observed  and  computed. 

(From  Dalzell.®®) 
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Fig.  11.11.  Amplitude  response  of  two  digital  filters. 
(From  Dedzell.®®) 


11.5  APPLICATION  OF  THE  fflGHER  ORDER  POLYSPECmA 


When  we  have  to  take  ink  account  nonlinearities  of  higher  order  than  quadratic,  we 
need  to  consider  the  polyspectra  of  higher  order  than  the  bispectrum.  For  example,  when 

Y(f)  is  expressed  by 

00  00  00 


=  / 


hiir)  Xit-r)  dr  + 


11 


^2(«l.“2)  X(t-Ui)  X{t-U2)du\dU2 


<-00 


^00  ~00 


+ 


111 


.00  .  00  .  00 


h3iquq2,q3)  X(t-qi)  X{t-q2)  X(t  -  qi)dqidq2dq3. 


(11.67) 


we  have  to  consider  the  fourth  moment  correlation  functions  and  their  three-dimensional 
Fourier  transforms,  which  are  called  trispectra.  In  this  case,  as  we  did  for  the  quadratic 
case,  we  can  calculate  the  statistical  values  as  summarized  in  the  following  paragraph. 

There,  the  variables  Xi,X2,-  •  ■  A;,  are  supposed  to  follow  the  joint  Gaussian  distri¬ 
bution,  with  £{An)  =  0,  then  as 

fpCi]  =  0,  E[XiX2X3-\  =  0,  E[XiX2  •  •  •  X53  =  0, -  (11.68) 


all  of  the  odd  moments  are  zero.  Further,  as  shown  in  Eq.  11.39 
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£[311X2X3X4]  =  Afi2M34  +  A/i3Af24  +  Afi4A/23.  (11.69) 

£[X  1X2X3X4X5X6]  =  M 12M  74MS6  Af  nMssM 46  +  ^^12^^36^/45. 

MijMiaMss  +  ^0^25^36  +  A/l3A/26Af45 
+  Afi4M23Af56  -t- Afi4Af25M36  +  M  14^26^35 
+  Afi5M34M26  +  A/l5Af32A/46  +  A/ 15^/36^24 
+  Mi6A!f34A/52  + Afi6Af35Af42  +  AfiftAf  32M45,  (11.70) 

where  m,>  =  £[X,  X*J.  These  characters  as  shown  m  Eqs.  11. 6»,  11.69,  and  11.70  were 
fully  utilized  in  the  calculations  of  the  following  together  with  the  symmetry'  properties  of 
hz  (tti,  U2)  /i3(  <7i.  92. 93)  in  terms  of  ui,  1/2;  9i.  92. 93- 

1 .  Mean  of  r(r),  my  =  £[y(r)] 


my 


00  00 

=  11  h2{Ui,U2)  R„{U2-Ui)  duidU2 
—00 

00  00  00 

~  j  j  ^2(Ui,U2)  j  5XX(h>)exp(-t(<WWl +0>«2))  do)  ■  duidu2 
—  »  —00  —00 

=/ 


H2{(0,-(o)  sxxUdi)  do). 


Equation  11.71  is  the  same  expression  with  Eq.  11.33’  for  the  quadratic  process. 
2.  Second  order  covariance  function,  cross  spectrum 

RYx(.r)  =  E[[Y(t)-mY]X(t-t)] 


(11.71) 


=  / 


hi(.r)Rxx{r-r)  dr 


+  3 


00  00  00 

ill 


—  00  —  00  —  00 


/»3(9i.  92. 93)  Rxxigi  -  9i)  Rxxir  -  93)  dqidq2dq^.  (11 .72) 


Therefore 


00 

1 


5yx(tw)  =  ^i(<y)^xx(<y)  +  3  /f3((Wi,-(Ui,m)  sxx(toi)  ^xx(n>)  ^o>i 


00 

I 


=  5xx(<w)  [Hi((u)  +  3  //3((0 1,-01, to)  Sxsdfoi)  rfoi].  (11.73) 
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The  term  SYx{o))/sxx{Oi>)  is  no  longer  equivalent  to  but  is  modified  by  some  term 

that  includes  the  third  order  nonlinear  frequency  response  function  Hi. 

3.  Second  order  autocovariance  functim,  linear  spectrum 


Ryy(J)  =  E[Y{t)  -  my][Y(t  +  T)  -  my] 
=  E[Y{t)Yit  +  T)]-m^ 


=  11  hi(si)hiiri)E\X(t-Si)Xit  +  t-ri)]dsidri 

«Q0  —00 
00 

^111/ 


h2(ui,U2)  h2{vi,V2)  E[Xit-ui)  Xit-U2)  X{t  +  r-vi) 

X  X(r  +  T-V2)]  du\du2  dv\dv2 


OD 

iiiai  hiipuP2,P3)h3{qu  qi,  qz) 


X  E\x{t-px)  X(t-p2)  X{t-pi)  X{t  +  r-qx)  X{t  +  t-q2)  X(r  +  T-^3)] 
dpidp2dp3dqidq2dqs 

(11.74) 


«.2 

—  Wy. 


We  take  the  Fourier  transform,  and  the  linear  spectrum  is,  after  manipulation. 


syritif  =  sxx(of) 


I 


ffi(a})  +  3  H3(w,W3,-q)3)  sxx(c03)  da)3 


+  2 


00 

1 


H2{0)- (02,(02) 


'Sxx{(0-(02)  Sxxi.(02)  da)2 


—  00 
00  00 


+  6 


H3{(0  -  (02,  -  <03,  <02,  <03)  j  ^  SxxkfO  -  <02  -  <03)  Sxxififi)  d(0'^3, 

(11.75) 


The  first  term,  the  term  of  sjodso)  of  this  Eq.  11.75  is  again  modified  by  the  cubic 
response 7/3,  as  was  Eq.  11.73  that  shows  syx(<o)/sxx(<o)  is  not  anymore,  but  is 

modified  by  the  effect  of  cubic  response  //3(a)i,a>2,<03).  Bedrosian  and  Rice*°  showed 
this  modiffcation  clearly  for  the  case  when  the  input  was  the  sum  of  sinusoidal  waves, 
and  showed  the  necessity  to  include  the  higher  order  terms  in  Voltera  expansions. 
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4.  Third  moment  cross  covariance  and  cross  bispectrum 

oe  Qo 

^K!aKTl,T2)  =  2  J  j  h2{Ui,U2)  Rxxi'^l-Ui)  Rxxi‘f2-t*2)  duidU2,  (11-76) 
—00 

therefore  Syxxi^ h <02)  -  1/^2)  Sxx((^i)  sxxi^^z)- 

If  we  use  ^-(f)  instead  of  y(r),  frcan  Eqs.  1 1 .56  and  1 1 .57  we  get 

=  2/?x3K*^i)  Rxxi'^i) 

Sx‘xx(.0)h^2)  =  2sxxi0il)  Sxxi(i>2) 


(11.77) 

(11.78) 

(11.79) 


as  we  did  in  the  preceding  section  for  Eq.  11.50,  inverting  the  variables  from  a)\,a}2  into 
0)1  -tW2,a>i  +tU2  , 

2  SyXX  (<Ui-a)2,(0l+0)2)  =  SYXX(^hO)2)-  (11.80) 

Therefore,  the  quadratic  frequency  response  function  H2  {a>\  a>2)  is  obtained  by 
SYXXiO)  1.(02)  Syxx((0 1,0)2) 


H2((0  1,(02)  = 


2sxx((0l)sxxi(02)  Sx2xxi(Ou(02) 
SYXX((Oi~(O2^0^1'^(O2)  ,  SYXX((0  1  ~  <02^  (0 1  (O2) 

SX^XX^O)!- (02,(0 1  +  0)2) 


(11.81) 


(11.81’) 


■SXjKflil)  SxxiO)2) 

These  are  the  same  as  Eqs.  11.49, 11.55, 11.58,  and  11.59  for  the  quadratic  process  and  it 
means  that  the  process  to  calculate  the  quadratic  response  Hz  (a>  1,0)2)  is  the  same  as  for 
the  quadratic  nonlinear  process,  even  for  this  third  order  nonlinear  process. 

5.  Fourth  moment  cross  correlation  and  trispectrum 

When  y(r)  is  expressed  by  Eq.  1 1 .67,  which  includes  the  third  order  nonlinear  for 
the  cubic  nonlinear  process  term,  the  fourth  moment  cross  correlation  can  be  obtained  by 
the  same  kind  of  manipulation  as  in  the  quadratic  nonlinear  process  as 

^yxxx(^i.i^2.^3)  “  X(f-Ti)  X(f-T2)  X(t-xi,)] 


oe  00  00 


-11/  ^XXi^-Ql)  ^XX(T2-92) 

dqidq2dqi.  (11.82) 


—  00  —  00  —  00 


Its  three-dimensional  Fourier  transform  is  called  the  trispectrum  and  is 

^lXoKo>l.<W2,a)3)  =  6  //3(<U  1,0)2, <03)  SXx(<Wl)  SxxiO)2)  Sxx^(Oi). 
If  we  use  X^{f)  instead  of  y(r),  then 

^x®xxx(^l»'^2>^3)  =  6i?X3K^l)^XX(T2V?XX('f3) 
^x^xxx(^l'^2r(03)  =  6sxx(0)i)Sxx(0)2)SXX((03)- 


(11.83) 

(11.84) 

(11.85) 
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Therefore,  the  third  order  nonlinear  frequency  response  function  is  obtained  as 


SYXXX^0i\),0>2A0ii) 
6Sxx(OJl)  Sxxito  2 


SyxxyltOi,  0)2,  (O^) 


(11.86) 


6.  AppUcation  of  the  trispectrum  to  ship’s  rolling  by  Dalzell. 

J.  F.  Dalzell*’  discussed  the  characters  of  higher  order  nonlinear  responses  in  Vol- 
tera  expansion,  especially  the  interferences  between  different  order  nonlinear  responses, 
based  on  E,  BecLosian  and  S.  O.  Rice***  and  also  tried  to  solve  the  problems  encountered 
on  their  numerical  computations.  Besides  Dalzell 's  trial^^^^  on  the  application  of  the 
bispectrum  analysis  to  the  added  resistance  of  ships  advancing  in  waves  (already  referred 
to  in  Section  11.4),  he  also  tried*’  an  application  of  trispectmm  analysis  to  the  problem  of 

nonlinear  ship’s  rolling,  expressing**  the  nonlinear  roll  damping  by  cube  of  roll  angular 
velocity. 

Dalzell  expressed  the  equation  of  motion  as 

X  [A,(y(t)p+B>{nr)p+C>{y(r)}’^  ^  (11.87) 

>=1 

Then,  using  the  so-called  incommensurate  frequency  technique, *°  the  relations  of 
=  l  —  3)  and  frequency  response  functions  Hi(tu),  H2{a>],a>2), 
HziwutozyO}})  were  derived. 

Setting  appropriate  values  for  Ai  to  C3,  and  using  the  simulated  series  of  wave 
heights  XiT)  that  have  Pierson-Moskowitz  type  spectra,  Dalzell  synthesized  the  roll 
response  Y{t)  as  shown  in  Fig.  11.12,  using  impulse  responses  hi{x),  hziixurz)  and 

obtained  through  H\(fii),Hz{o>uo>-2)  and 7/3(0)  1, 02. t«>3)  expressed  by  Aj,  Bj, 
and  CjU  =  1-3). 


Y1  (t) 


Fig.  11.12.  Simulated  time  histories  of  linear  random  excitation  X(()  and  nonlinear 
response  Y(f),  and  its  components  Y2(t),  and  Vaioa,  *  1.0. 

(From  Dalzell.'*^) 

Here,  T^f),  KaCO.  and  Y^it)  show  the  first,  second,  and  third  terms  of  the  Voltera 
expansion  of  Y{t)  as  Eq.  11.67. 

Figures  11.13  and  11.14  show  the  first  and  second  order  impulse  response  functions 
hi(t)  and  >12(^1,  r2)  in  the  form  of  the  weighting  functions  gj,  ^  and  Fig.  11.15  shows  a 
portion  of  the  third  order  impulse  response  function  />3(rj,  t2,  h)  at  the  sequence  of  val¬ 
ues  in  the  form  It  is  interesting  that  impulse  responses  are  obtained  quite  beautifully 
by  this  analysis. 


Fig.  11.13.  Truncated  lirtear  discrete  kernel  j’j. 
(From  Dalzell.'*^) 
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Figure  11.16  compares  the  observed  and  the  predicted  second  order  moment  spectra 
and  also  U\,  Ui,  and  U-^  correspond  to  the  first,  second,  and  third  terms  of  Eq.  11.75.  It  is 
rather  surprising  that  the  observed  value  can  be  almost  fully  explained  theoretically  by 
Eq.  11.75.  It  is  also  interesting  to  find  in  the  upper  second  figure  of  Fig.  11.16,  that  the 
linear  estimated  spectrum  (shown  by  solid  line)  is  pretty  largely  distorted  by  the  existence 
of  cubic  frequency  response //3  as  are  shown  by  Eq.  11.73  and  by  the  first  term  of  Eq. 
11.75,  and  is  modified  into  IJ\  (o)  shown  by  the  dotted  line  in  the  same  figure. 


Fig.  11.14.  Truncated  quadratic  discrete  kernel 
(From  Dalzell.''^) 
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FREQUENCY,  rad/sec 


Rg.  11.16.  Observed  and  predicted  spectra  and  the  components  of  spectrum  of 
nonlinear  response  of  the  simulated  system,  nominal  a,  =  1 .0. 

(From  DalzeU.^^) 

7.  Expansion  to  higher  order  nonlinear  process. 

By  the  same  procedure  as  previously  used  for  quadratic  and  cubic  nonlinear  pro¬ 
cesses,  we  can  go  on  to  higher  order  nonlinearities.  For  example,  from  Brillinger,*^ 

^YXX-  •  '  '  ' ^m) 


H„i(0i,(02,-  ■  (Om)  = 


mlsxxitoi)  sxx(o>2)  ■  •  •  sxxi<t>m) 


(11.88) 
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CHAPTER  12 


PROBABILISTIC  CHARACTERS  OF  NONLINEAR  RESPONSE  PROCESS 
12.1  INTRODUCTION 

The  stochastic  or  random  process  can  be  expressed  as  a  function  of  time  and  also  as 
a  probabihsitc  measure  x(r,s).  Until  Chapter  11,  however,  we  have  discussed  the  charac¬ 
ters  of  the  process  mostly  in  the  time  aspect  and,  based  on  the  ergodicity,  dealt  with  the 
correlation  functions  and  spectra  and  their  statistical  characteristics.  Especially  in 
Chapters  9, 10,  and  11,  the  effects  of  nonlinearities  on  these  characteristics  were  smdied. 

In  this  chapter,  the  probability  distributions  that  characterize  the  response  process 
will  be  discussed  briefly  for  reference.  The  main  topic  in  Part  m  is  the  nonlinearity  of  the 
process.  However,  general  considerations,  including  the  linear  cases,  will  be  discussed 
first. 

Generally,  the  probabilistic  character  of  the  random  process  X(r)  can  be  defined 
completely  by  the  series  of  probability  distribution  density  functions  as 

'•  the  probability  that  X(t)  is  in  the  range  a:i  ~  Xi  dxi 

at  time  t\\ 

P2i.X\,t\;,X2,t2) ;  the  probability  that  X{t)  is  in  the  range  Xi  x\ -¥(1x1 

at  time  t\ 

and  in  X2  —  a:2  +  dx2  at  time  ti, 

P3{x\,t\-,X2,t2',xi,ti) :  the  probability  that  X{i)  is  in  the  range  X\  —  xi  +  dxi 

at  time  ti, 
in  j:2  —  X2  +  dx2  at  time  t2, 
and  in  X3  --  X3  -t-  dx^  at  time 

In  the  same  way  p„  is  defined  for  n=4,  S  . . .  . 

Here  i.  p„  Ss  0 

ii.  p„  is  symmetrical  for  xi,rj;  JC2,r2;x3,r3; . . . 

iii.  the  marginal  probability  density  function  pm  is 
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00  00 


^^m-^-ldXm-^2  •  •  •  ^n- 


(12.1) 


n  —  m 


As  was  mentioned  in  Section  2.1,  when  these  probability  density  functions  pn  stay 
the  same  when  the  time  is  replaced  by  rj-t-T  (T:  arbinary),  this  process  is  called  station¬ 
ary.  Then  pi(JCi,  t\)  is  independent  of  time,  and P2  depends  only  on  the  time  difference 
rwi.  Here  the  concept  of  the  Markov  process  that  is  used  to  classify  the  randcsn  process 
probability  is  first  introduced. 

From  the  characteristics  of  these  distribution  functions,  the  processes  are  sometimes 
classified  into  three  groups:  (1)  pure  random  processes,  (2)  Markov  processes,  and  (3) 
general  processes. 

For  a  pure  random  process,  since  the  value  x,  at  time  r,  is  independent  of  any  values 
of  jc(r)  at  any  other  times  r,_i,  r,_2, . . .  ti.  ti.  this  process  can  be  defined  completely  by 

PiOk.ti),  because  any  higher  order  joint  probability  distribution  fimction  is 

n 

Pn(Xi,ti;X2,t2  .  .  .  x„,f„)=  n  pi(.Xi,ti)  .  (12.2) 

J=1 


Accordingly,  if  only  piiXi,ti)  is  given,  all  other  joint  distribution  functions  can  be  derived 
easily. 


12.2  MARKOV  PROCESS 


When  the  process  is  not  pmely  random,  but  when  in  addition  to  p\  like  p\(.Xi,  r,j  and 
if  P2(Xj-\,tj-\',Xj,tj)  is  necessary  to  express  the  probability  characters  of  the 
process,  the  process  is  called  a  Maikov  (or  Markovian)  (linear)  process.  The  condition 
that  the  probability  distribution  density  functions  pi(Xi_i,  r,_i)  and  p2(JC/-j.  O-iI  xj,  tj)  are 
given  is  the  same  as  the  condition  that  the  conditional  probability  distribution  density 
functions Pciixj,  tj  -  r^i Ixjl-i)  are  known.  Because  the  conditional  probability  density  func¬ 
tion  PaiXj,  tj\x}-it^\)  is  the  probability  distribution  drasity  function  of  this  process  that  x 
is  in  the  range  Xj  —  xj  +  dxj  at  time  tj,  under  the  condition  that  the  value  of  this  process 


was  in  the  range  of  +  dxj_i  at  time  r^i,  that  is  {tj-tj-i)  prior  to  tj,  and  is 


expressed  by 

PcMp  O-i)  =  tj-t^i\x^i)  = 


P2{Xj,tj;xj.itj^i) 

Piixj^utj^i) 


(12.3) 


Here,  from  t"  s  characteristics  of  the  probability  distribution  density  function, 
i.  pcj{xj,t\xj-\)  ^  0 
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m 

ii.  ^  pcj(xj,t^j_i)dxj^\ 


iii.  P\ixj,tj)-  j  pi(x^utj-i)Pc2(Xj,tj-t^i\x^i)dxj.i. 

•~.CC 

Generally,  the  Markov  process  is  defined  as  follows,  using  the  conditional 

probability - when  the  probability,  thatX(r)  is  in  the  range  xj  —  xj  +  dxj  at  tj,  under 

the  condition  that 

X(f)  is  Xj-i  -  Xj_i  +  dxj.u  at  r^i, 

Xj-2  -  xj.2  +  dxj.2,  at  r^2. 

X2  -  r2  +  d*2,  at  t2, 

^  Xi  —  aci  +  dxi,  at  rj, 

is  determined  by  the  conditional  probability  that  X(t)  is  xj  xj  +  dxj,  under  the  condi¬ 
tion  that  at  only  one  preceding  time  step  r  =  r^i,  ;c(r)  is  xj^j, - then  this  process  is 

called  a  Markov  process.  Namely,  if 

Pc(.Xji  ^j\Xj—lt  •  •  •  »  X2)f2’Xi,ti} 

=  PciiXj,  tj\xj-u  (12.4) 


this  process  X(f)  is  a  Markov  process. 

From  Eq.  12.3  and  Eq.  12.4,  generally 

tyi,  ...  t||*]_) 

X  •  X2,t2\xut{).  (12.5) 


Therefore,  by  the  same  relation, 

1»  tn_j,  .  ,  .  X2jt2jXijt\) 

X  Pc2iX2,  t2\xi,  ti)pi{Xu  h).  (12.6) 
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From  this  expression,  the  term  Markov  chain  is  used.  The  conditional  probability 
function  is  also  called  the  transitional  probability  function. 

Also 

OD 

~  f  1»  ^/i— 1»  2,^«— 2)  1 


and 

P2i^ni  ^nt  ^n—2t  ^n-2^  ~  Pc'J^^nt  2»  ^n-2)  PO^n-2'  ^n-2^ 

1»  ^#1— ^n—2^  ^#1*2)  “  Pc^^iti  ^n*  2,^w^2^ 

^  Pi.Xn-2^  tn-2) 

therefore 

00 

2)  ~  |"  Pc(-^n>  ^n—lt  fn—l^n—2’ ^n—2^^^rf~l  (12.7) 


this  Eq.  12.7  is  called  a  Chapman-Kolomogorov  equation  or  Smoluchowski  equation  for 
the  transitional  distribution  of  the  Markov  process.  It  implies  that  to  go  frcan  x„_2  at  time 
tn^2  to  x„  at  time  tn,  the  path  Xn-i  at  time  f,^i  is  not  important.  We  can  take  any  path  to 
go  to  x„.  This  characteristic  is  used  to  derive  the  Fokker-Planck  equations,  shown  in  the 
next  section. 

For  example,  the  AR  model  of  the  fust  order  AR(1)  as  was  treated  in  (Thapter  S  is 

X{t)-aXit-l)  =  €{t),  (12.8) 

where  n  is  a  constant,  €(f)  is  a  pure  random  {X'ocess  (a  Markovian  linear  process),  and  so 
X{t)  is  statistically  determined  by  the  value  X(r-1),  that  is,  the  value  of  X(r)  at  one  preced¬ 
ing  time  step.  Accordingly,  X(r)  is  a  Markovian  process. 

12.3  GENERAL  PROCESS 

If  we  need  the  values  of  a  process  not  only  at  one  preceding  time  point  r,_i,  but  at 
two  or  more  preceding  time  points,  i.e.,  not  only  p\  and  p2,  but  also  higher  order  joint 
probability  functions  pj,  p4, . . .  the  process  is  called  a  general  process.  However,  among 
general  processes  X{t),  some  can  be  inverted  into  vector  Markov  processes,  combining 
with  some  other  variables  and  introducing  the  concept  of  the  state  space  (Markovian). 

Putting  aside  the  concise  theory,  for  example  for  the  second  order  autoregressive 
model  AR(2)  as 


X{t)-axX{t- l)-a2X(t-2)  =  €(t). 


(12.9) 


we  can  introduce  the  vector  process  X(t)  with  two  elements  Xi,X2  as 

fx2(t)=X(f) 

I  Xiit)  =  a2Xit- 1)(=  a:^2it- D- 


Then  Eq.  12.9  can  be  written 


^^2l  r^i(^-i)i  ^  [oi 
X2(oJ  [l  nij  [X2(/-1)J  [ij 


(12.10) 


(12.11) 


Xit)  X(r-l) 

Here  setting 


Eq.  12.11  becomes 

X(t)  =  a  Xit-1)  +  e(f) .  (12. 1 2) 

This  transformation  shows  that  AR(2)  can  be  inverted  into  a  Markovian  linear  process  or 
a  vector  process  X(t)  =  [Xi(r),X2(r)]'. 

By  the  same  procedure,  the  autoregressive  process  of  order  n,  AR(n),  or  more 
generally  the  autoregressive  moving  average  model  ARMA  (n,  m),  can  be  shown  to  be 
invertible  into  a  Markovian  process,  introducing  die  matrix  of  inverting  functions. 

The  characteristics  of  a  Markovian  process  contribute  to  the  derivation  of  distribu¬ 
tion  functions,  applying  the  Fokker-Planck  equation,  and  also  for  the  application  of  the 
Kalman  filters. 

12.4  THE  FOKKER-PLANCK  EQUATION 

To  show  the  behavior  of  the  transitional  probability  density  function  pc  of  a  stochas¬ 
tic  process  as  a  Markov  process,  formally  a  linear  second  order  partial  differential 
equation  called  the  Fokker-Planck  equation  is  used.  This  equation  was  developed  by 
Fokker  (1914)  and  Planck  (1917),  to  indicate  the  Brownian  motion  of  molecules,  and  it 
has  also  been  utilized  as  the  equation  of  motion  to  show  the  behavior  of  the  nansitional 
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probability  density  function  of  a  stochastic  process.*^-*^  This  equation  is,  however,  not 
solved  analytically  in  general,  except  for  a  few  special  cases,  so  special  considerations  are 
usually  demanded  in  using  it. 

Here  the  conditional  or  transitional  probability  density  function  Ar|>’)  is 

expressed  merely  is  pc,y  being  the  value  of  process  X(r)  at  a  preceding  time  At.  The 
derivation  of  this  equation  is  given  by  Caughey.^^ 

The  Fokker-Planck  equation  for  this  transitional  probability  function  is  in  the  form 


dpc 

at 


-4-D^‘\x) 


+ 


iJi 

2  Bx^ 


D^^\x) 


(12.13) 


Here  D^^\x)  and  D^^Hx)>0  are  called  the  drift  coefficient  and  diffusion  coefficient, 
respectively,  and  are  related  to  the  first  and  second  moments  of  this  distribution  as 

00 

D^^)(jr)=  lim  \  {y  ~ x)pc(x,  A^y)dy  (12.14) 

Ai  — 0  At  j 


D^^\x)^  lim  -J-  \  (y-xfpcix,At\y)dy.  (12.15) 

A/  —  0  At  j 
*00 


Generally  these  are  functions  of  time  r. 

As  was  stated  before,  when  the  general  process  is  inverted  into  a  vector  Markov 
process  .¥(/)  of  N  dimensions  as  Jr(r)  =  [Xi(t),X2(t)  .  .  .  A)v(/)]’,  the  Fokker-Planck 
equation  is  more  generally  in  the  form 


dpc 

at 


a  1  ^ 

— £>!*>(  + 

Bxi  '  ^ 


2.^1  Bxidxj 


D'ihx) 


Pc 


(12.16) 


where 


d\\x)=  lim  ^ 
A/-»0  Af 


OD 

f  ^ 

J  Cyi-J^i)Pc(x,Af|y)n  dyi 


(12.17) 


Df{x)=  lim^ 

Af  — 0  At 


00 

r  N 

(y.  -  Xi)(yj  -  Xj)pc{  X,  Ar|  y )  n  dyi 

J  1=1 


(12.18) 


X,  y  are  the  values  of  process  X(t). 
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Solving  Eqs.  12.13  or  12.16  gives  the  transitional  probability  functicHis.  Since  these 
equations  are  solved  analytically  only  for  some  restricted  types,  several  methods,  such  as 
a  simulation  method,  transformation  into  a  Schradinger  equation,  numerical  integration, 
potential  method,  and  so  on,  are  used  to  obtain  the  solution. 

When,  by  the  elapse  of  time,  the  transitional  probability  function  pdx,  r|y)  tends  to 
p{x)  as  its  limit  and  becomes  stationary,  independent  of  time  and  its  initial  value,  then  in 

the  Fokker-Planck  Eqs.  12.13  and  12.16,  setting  t-*  oo  _  q, 

df 

N  N  2  ^ 

T  Z  Z  0,  (12.19) 

2  i=i  j^i  oXidXj^  J  y_j  dXi 

where  p(x)  obtained  as  the  solution  of  this  equation  is  an  N-dimensional  joint  distribution 
related  to  the  N-dimensional  state  space  for  this  vector  process  X(t). 

Under  some  considerations  and  restrictions,  these  methods  can  be  expanded  to  deal 
with  nonlinear  processes. 

For  example,  following  Caughey,^^  consider  a  nonlinear  oscillation  with  nonlinear 
restoration  under  Gaussian  (mean  =  0)  excitation  as 


X(f)  +  /?X(0  +  F(X(0}=/(r) 

(12.20) 

F[/(f)]  =  0 

(12.21) 

EWi)  = 

(12.22) 

here  Wo/2  shows  the  white  spectrum  of  the  excitation,  and  6  is  the  Dirac's  delta 
function.  From  the  transform 

•  .  (12.23) 

X2{t)=Xit) 

and  introducing  the  vector  process 

X{t)  =  [Xi(f),  ^2(0]'  =  [X(0,  x\t)r,  (12.24) 

from  Eqs.  12.20  and  12.23 
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Xi(t)=X2(t) 

X2(t)  =  -fiX2(t)-F{Xiir)]+M. 


(12.25) 


Coefficients  DP\x),and  D^^\x)  for  the  Fokker-Planck  equations  are,  when  we 
insert  Eq.  12.23  into  Eqs.  12.17  and  12.18, 


-*0  ^ 


A/  — 0 


(1)  £[AX2] 

D2  =  lim 


(12.26) 


At 


^0  Ar 


=0 

A/-0  Ar 


Z)\Y=  lim 


D^22  ~ 


£[AXiAX2] 


Ar-*0 


A/ 


(12.27) 


At  —  O 


£:[Ayi] 

At 


Substituting  £q.  12.25  for  EIAX2]  and£[AA(^  and  using  r  as  a  dummy  variable  in 
r  gives 


D2  ^  =  lim 
A/  — 0 


f+A/ 


j-^X2(r)-F(Xi(f)))A/+  j  j{t)dt 


At 


=  -^X2-F{Xi) 


(12.28) 
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=  lim 
A/  — 0 


=  lim  £ 
A/  — 0 


/+A/ 


-pX2{t)-F{Xiit))\^i+  j  f{t)dt 


At 


i+Ai 


[-PX2  -  FiXifAt  +  2[-pX2  -  £(Xi)}  I  Ar)dt 


+  lim  £i 
A/  — 0 


t+A/  t+Ai 

■^11  .^Ti)^T2)A-j^/r2 


2 


(12.29) 


Now,  the  process  is  stationary,  so  with  Eq.  12.19,  the  Fokker-Planck  equation  becomes 


(12.30) 


This  is  called  the  stationary  case  for  Kramer’s  equation  and  has  been  solved  by  several 
scholars.  Following  Caughey-Wu,  the  solution  of  Eq.  12.30  is 


pixi,X2)  =  p{x,x)  =  C  exp 


Wo 


0 


(12.31) 


Here  C  is  the  coefficient  for  normalization  and  t  is  a  dummy  variable  for  X|.  Equation 
12.31  is  in  the  form  of 


p(x,x)  =  C  exp 


-4^ 


Wo 


(12.32) 


where  £  is  the  total  energy  per  unit  mass  of  this  system 
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From  Eq.  12.31 


p{x,x)  =  pix)pix). 


and 


P(i)  =  exp^^i^ 


(12.33) 


(12.34) 


(12.35) 


pix)  -  C  exp 


H^o 


(12.36) 


we  find  the  probability  distribution  density  function  of  the  velocity  x  is  a  Gaussian. 
More  concretely,  for  a  nonlinear  spring  system,  FjX(f)j  in  £q.  12.20  is 

F{Xi(f)}  =  F(X(0}  =  +  eg{X(f)}]  (12.37) 

and  the  equation  of  oscillation  Eq.  12.20  is 

m  +pX{t)  +  (O^Xiw)  +  =yi0.  (12.38) 

Here  for  hard  spring  type,  e  >  0,  and  6  is  on  the  order  of  x(r)/g(x(f)),  g{x(r))  = 
-^{-x(r)),  and  as  bc(r)l>0,  x(r)g{x(r))  >0.  Nowwedefineeuoasthe 
undamped  (when  =  0)  natural  frequency  of  oscillation. 

The  mean  square  value  of  this  oscillation  is 
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OD 


I 


jrp(jc,  x)dxdx 


00 

=  I  j^xjjc  +  e^U)]  -  jcegU)  jpuWj:. 


(12.39) 


From  Eq.  12.36  and  Eq.  12.37 


pix)  =  C  expo 

►  =  C  exp 

1 

2 

—  +  eG(x) 

■ 

0 

..  1 

• 

G{x)  = 


For  a  corresponding  linear  system,  cr^ ; 


Wq 

4^0)1 


^2  _ 


For  the  linear  system,  c  =  0;  therefore  from  Eq.  12.39, 

00 

E[X\t)]  «  j  xegix)  p{x)dx 
~00 

*  g(^0}]. 


(12.40) 


For  a  bard  spring  system  the  mean  square  of  the  displacement  is  always  smaller  than 
it  is  for  the  corresponding  linear  system. 

For  the  so-called  Duffmg  type  system,  g{^(t)}  =  X^{t),  and  the  equation  of  motion 
is 


m  +  ^m  +  W  {^(0  +  6X^(0  j  =  AO.  (12.41) 
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Therefore  from  Eq.  12.40 

E[XHt)]  «=  al~eE[X^(t)l  (12.42) 

For  the  Gaussian  process  E[X\t)]  =  3a/.  Therefore  in  this  case,  the  order  is  also  in 

E[XHt)]^  a^-Seai  (12.43) 

It  is  interesting  to  find  that  this  approximation  is  the  same  type  as  Eq.  10.17 
obtained  through  the  equivalent  linearization  method  and  Eq.  10.50  obtained  through 
the  perturbation  method. 

12.5  PROBABILITY  CHARACTCRISTICS  OF  AMPLITUDES, 

MAXIMA  AND  MINIMA 

S.  O.  Rice®  has  shown  that,  when  the  joint  probability  density  function  of  x  and 

X,  the  values  of  a  stochastic  process  X(r)  at  time  r,  and  its  derivative  {d/dt)X{t),  i.e., 

p(x,i),  are  known,  the  frequency  of  occurrence  of  any  threshold  value  crossing  or  zero 
crossing  is  easily  derived. 

For  example,  the  expected  value  of  the  frequency  of  occurrence  of  the  process  X{t) 
for  the  upward  crossing  of  a  threshold  of  level  a,  £[N+(a)],  is 

00 

£[N+(a)3  =  j  xpia,x)dx.  (12.44) 

0 


If  x{t)  andi(r)  are  independent,  p(x,i)  =  p(x)  •  p(i),  therefore 

E[iV^.(a)]  {pjx)]  x=a 
E[N^m^  \pix)]^' 


(12.45) 


When  the  power  spectrum  is  narrow  banded,  the  amplitude  of  the  sample  process 
varies  slowly,  just  like  the  envelope  of  sine  waves,  and  ps  the  result,  the  process  is 
assumed  to  have  an  extreme  positive  value  Op  at  each  cycle  of  this  sine  wave. 

Then  the  probability  distribution  is 


PpiOp  >  a)  = 


E[N^ia}] 

E[N40)] 


=  l-P{ap  <  a). 


(12.46) 


Therefore,  the  probability  density  distribution  is 
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X  dP 
Pp(a)  =  — =  - 


_  f  1  )  I  d(E[NM)  I 

\£[A?^(0)]j|  da  j 


(12.47) 


When  X{t)  and  X(r)  are  independent, 


Pp(a)  =  - 


dp(x) 
dx  X 


(12.48) 


Expanding  this  kind  of  relation,  Longuet-Higgins,**  and  Cartwright  and  Longuet- 
Higgins,*^  developed  the  well-4aiown  results  for  the  distribution  function  of  extreme 
values  of  a  general  process  with  various  bandwidths. 

The  probability  distribution  density  function  for  extreme  values  of  a  process  X(r), 
or  I1.I2.  • .  ■  normalized  by  the  standard  variation  of  the  original  process  <7,,  as 
1/2 

- 1  was  derived  as. 


1  i  _li  f  _£i 

=  ^  ^  j  e  ^  du 


(12.49) 


as  shown  in  Fig.  12.1,  where  e  is  the  so-called  bandwidth  parameter  of  the  spectrum 


(mom4-wi5) 

m{/n4 


(1250) 


m„  being  the  moment  of  spectrum 


’"'J 


m„=:  j  a>''s{(o)d(o. 


(12.51) 


When  e  0,  as  for  a  narrow  banded  spectrum,  p(0  becomes  a  Rayleigh  distribution  as 
is  well  known. 


I  0  t<0 


(12.52) 


or 


p(D  =  V 


e  S  0 

^<0 


(12.52’) 


At  the  other  extreme,  when  e  =  1,  as  when  ripples  are  superimposed  on  slowly  varying 
waves. 


pO  = 


1 


(12.53) 


1 

p(D=y^  ^  =  (12.53’) 

P(^)  becomes  Gaussian  and  it  is  the  same  as  the  Gaussian  distribution  that 
governs  the  original  process  x(r). 

When  e  is  between  0  and  1 ,  /K5)  is  given  by  Eq.  12.49  and  is  between  the 
Rayleigh  and  Gaussian  distributions,  as  shown  in  Fig.  12.1,  and  is  now  popular  for  us. 


Rg.  12.1.  Probability  distribution  density  function  of  extremes. 
(From  Cartwright,  Longuet-Higgins.''®) 


From  this  distribution,  the  expected  highest  value  of  1/n  and  the  expected  highest 
values  of  N  independent  samples  have  been  derived  as  functions  of  e.  The  derivation  will 
not  be  referred  here,  as  they  are  well  known. 
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Equations  12.46  and  12.47  can  be  applied  to  a  nonlinear  process  as  long  as  the 
process  is  narrow  banded.  Crandall^  analyzed  a  hardened  spring  oscillation  system,  the 
same  as  the  one  analyzed  by  Caughey^^  utilizing  the  Fokker-Planck  equation,  as  was 
shown  in  Section  12.4.  Equation  of  motion  is  by  Eq.  12.20, 


X(f)  +  ^X(r)  +  F{X(/))=y(r). 

He  obtained  the  average  period  T(a),  the  probability  distribution  function  of  the  peak  val¬ 
ues  Pp(a),  and  the  probability  distribution  function  of  the  envelope  pe{a)  as  functions  of 
the  amplitude  a  by  introducing  the  potential  energy  per  unit  mass 


V{x)  = 


0 

The  solution  of  the  Fokker-Planck  equation,  Eq.  12.31  {whtnfix)  is  Gaussian)  is 

p{x,x)  =  C  exp 

where  C  is  a  normalizing  constant  that  makes 

00  00 


1 

y+V(x) 

► 

♦ 

’  m 

1 

—  00 


J 

—  00 


pix,  'x)  dx  dx=l. 


(12.54) 


(12.55) 


The  results  are,  as  for  the  envelope  process  a(f), 

Via)=^^  +  V{x), 


Pia 


a  [2|V(a)-V{x))]I 

t<a)  =  A^dx  I  p{x,'x)  dx 


(12.56) 


(12.57) 


I 
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a 

Peia)  =  =  4F(fl)  j [l[V{a)  -  F(jc))]^  p^x,  [2[Via)  -  nx))]^)  dx.  (12.58) 


Through  manipulation,  the  following  were  derived; 


Pe{a)  =  C  •  T(a)  •  Fia)  exp  <  - 


Via) 


=  Ppia)v^  Tia)  (12.59) 


and  also. 


Ppia)  =  F(a)  •  exp 


V(fl) 


(12.60) 


where  vj  is  the  expected  number  of  threshold  crossings  per  unit  time  of  the  level  x  =  0 

with  positive  slope,  and  can  be  derived  as  Vq  =  Ca/ ;  r(a)  is  the  average  period  as  a 
function  of  amplitude  a. 

Crandall  showed  the  general  solution  for  the  following  two  nonlinear  cases; 

i)  for  a  hard  spring  Duffmg  type  system, 

F{X(r)}  =  (oi  {X(f)  +  eX^Coj,  (12.61) 

ii)  for  a  set-up  spring  system,  as  in  Fig.  12.2, 

F(X(/)}  =  (o^  [:y(/)  +  e  sgn  X(r)),  (12.62) 
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Fig.  12.2.  Displacement-force  relation  of  seMjp  spring. 


as  are  shown  in  Figs.  12.3  and  12.4.  Herea^^  is  the  standard  variation  of  the  linear  system 
when  €  =  0.  These  figures  show  that  the  probability  distribution  of  extreme  Pp 

varies  considerably  with  the  extent  of  nonlinearity,  expressed  by  the  value  of  c . 


Fig.  12.3.  Probability  distribution  density  of  the  extreme  (peak) 
values  of  Duffing  type  oscillator. 

(From  Crandall.®^) 
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Fig.  12.4.  Probability  distribution  density  of  the  extreme  (peak) 
values  of  oscillator  with  set-up  springs. 

(From  Crandall.®^) 

12.6  APPUCATION  OF  THE  FOKKER-PLANCK  EQUATION  FOR 
THE  ANALYSIS  OF  SEAKEEPING  DATA 

As  has  already  been  mentiooed,  the  Fokker-Planck  equations  have  been  solved  only 
for  a  restricted  number  of  cases,  and  this  has  made  the  applicability  of  this  equation  rather 
difficult  in  many  engineering  fields.  For  example,  this  method  was  applied  to  the  nonlin¬ 
ear  system  with  nonlinear  restoring  terms  and  with  Gaussian  white  excitation,  as  shown 
in  the  preceding  section,  but  the  method  has  been  considered  inapplicable  to  the  system 
with  nonlinear  damping  or  with  colored  noise  excitation.  A  few  efforts  have  been  made  to 
overcome  these  difficulties,  for  example  by  J.  B.  Roberts.*’  His  work,*®'®*  especiaUy  in 
the  analysis  of  nonlinear  seakeeping  ^ta,  will  be  summarized  briefly. 

12 .6.1  Nonlinear  Analysis  of  Slow  Drift  Oscillation  of 
Moored  Vessels  in  Random  Seas 

Utilizing  the  known  characteristics  of  the  Fokker-Planck  equation,  Roberts*® 
analyzed  the  statistical  behavior  of  the  nonlinear  slow  drift  motion  of  moored  vessels. 

Assuming  that  the  waves  7j{t)  are  narrow  banded  with  band  width  parameter  e,  and  that 
the  drifting  force  Z)(f)  can  be  regarded  as  proportional  to  the  square  of  the  wave  height,  he 
modiffed  the  expression  of  wave  height  ^(r)  and  drifting  force  £>(r)  to 

ff(t)  =  H(t)  cos  [a>ot  +  a(t)]  =  hA{t)  cos  [(Oot  +  a(t)] 

=  /i|^|a^(r)  +  j  cos  [toot  +  a(0] 

=  h[a{t)  cos  (0(f-b{t)  sin  aioT],  (12.63) 
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berea>o  is  the  peak  frequency  of  the  narrow  banded  spectrum  of  the  wave,  and 

D(t)  =  a[H^(t)]  =  a'[cr(t)  +  tHO]  =  D  +  /t'i(r),  (12.64) 

where  //(f).  hA(t)  are  the  amplitudes  of  the  envelope;  a(r)  and  b(r)  are  components  of 
A(r);  a{t)  is  the  phase  lag;  ^ is  the  mean  drifting  force;  and|(r)  is  the  white  noise,  as  will 
be  assumed  later. 

The  equation  of  swaying  motion  of  the  vessel  was  expressed  as 

(M  +  mix  +  Fix,x)  =  D(t)  (12.65) 


or 

X  ^fix,x)  =  ^{a\t)  +  h\t)\  (12.66) 

where  (A/  +  m)  is  the  virtual  mass  of  the  vessel.  Then  with  a(r),  b{t)  as  the  output  of  white 
noise  ^(0  through  linear  filters,  the  equation  of  motion  was  modified  to 

i  +/(2  y=  ^(0-  (12.67) 

The  term  z  is  a  vector  Markov  process  with  four  elements  [x,x,  a,  b],  and  the  Fok- 
ker-Planck  equatitm  of  this  four-element  variable  z(r)  was  then  derived.  Then  since 
A(r)  can  be  generated  as  the  output  of  a  nonlinear  first-order  system  with  a  white  noise 
input  ^(f),  the  process  was  inverted  into  a  three-element  vector  Markov  pro¬ 
cess,  y(r)[x(f),xi(r),  A(f)],  and  the  Fokker-Planck  equation  for  this  process  was  also 
derived. 

However,  the  solution  of  the  multidimensional  Fokker-Planck  equation  has  a  num¬ 
ber  of  difficulties,  so  Roberts  advanced  the  approximation  further,  and  expressed  the 
drifting  force  as 

D(t)  =  D  +  k’^(t),  (12.68) 

the  sum  of  the  mean  drifting  force  U,  that  can  be  approximated  as  a'e,  plus  a  white  noise. 

Under  diese  approximations,  the  equation  of  swaying  motion  is 

x  +  gix,x)  =  d  +  k^it). 


where  d  = 


D 

(M  +  m) 


a'e 

(M  +  m) 


=  /3e. 


(12.69) 


The  two-dimensional  Fokker-Planck  equation  was  derived  as 
dt 


2  az,2 


(12.70) 


where  Z\  =x,  Z2-  x,  and  p  is  the  transitional  probability  density  function. 

The  stationary  solution  of  p  is  obtained  by  (dp/ dt)  =  0.  As  g(x,  x)  in  Eq.  12.69.  in 
its  general  form 

g(x,  x)  =  yF(x,  x)  +  Gix),  ( 1 2.7 1 ) 


is  used. 


Then  the  joint  probability  density  function  of  swaying  motion  p(x,  x)  is  the 
solution  of  Eq.  12.70,  when  (dp/dt)  =  0, 

pix,x)  =  C  exp[— U(x)-dx  y  I  (12.72) 


Y  ^ 

f  * 

X^ 

ejqj 

Y+U(x)-dx 

► 

where 


X 


G{k)dX 


(12.73) 


is  the  potential  energy  for  restoring  forces,  C  is  a  normalization  constant, 
and 

V 

0 


Here 

^(v)"'^A^  1  ^(^)^2(v-C/(x)a[x  (12.75) 

R 


C(v)  =  -^  f  /v-U(x)dx  (12.76) 

A(v)  J 

R 
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A(v) 


-1  f  ^ 

"2  J  ^v-U{x) 


(12.77) 


The  integration  range  R  is  such  that  U{x)  <  v.  Especially  for  Duffing  type  oscillation  the 
equation  of  motion,  Eq.  12.69,  is  expressed  by 


3r  +  2t»: + j: + ^  l(t). 

(On  (On 


(12.78) 


Here  r  is  replaced  by  nondimensional  time  r,  r  «  o)^,  o>„  being  the  undamped  natural 
frequency  in  the  linear  case  (2  =  0),  and  the  nondimensionalized  equation  is 


x'  +  2  ^i'+x'  +2*  x'^  =  </*  +  2  2:^/2^(r), 
where  the  differentiation  is  with  respect  tor  and 


i'  =  — , 

Ox  k 


(12.79) 


?(1+4€*2) 

1“"/ 

e* 

1/2 


.e*  = 


(On 


The  joint  daisity  distribution  for  p(x',  x')  and  the  probability  density  distribution 
function  p(x')  is,  from  Eq.  12.72, 


p(x',  x')  =  C  exp 


fi’2 

< 

x'^  A  *  x'^ 

/ . . 

T  T  ■  U  «4 

2  4 

4 

(12.80) 


and 


00 

p(x’)  =  I  p{x',  x’)dx'  =  C  exp 


x'2  A*x 
+ 

2  4 


.  (12.81) 


Here  C  is  a  coefricimit  for  normalization. 
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QC  00 

j  j  p(x',x’)dx’di' =  1. 


Roberts  calculated  numerically,  for  a  Duffing  type  system,  with  various  and 

€  *  and  compared  them  with  the  simulated  results.  Figure  12.5  shows  an  example  of  his 
results.  He  compared  p(x)  calculated  by  Eq.  12.81  under  the  white  noise  approximahoo 
with  the  result  obtained  by  simulated  data  and  found  that  the  agreement  is  good  and  quite 
different  from  the  Gaussian  distribution  for  A*  =  0.  Once p(x)  of  the  displacement  x{r)  is 
determined,  p(r),  the  probability  distribution  of  tension  on  the  mo<»ing,  can  be  derived  as 


P(T) 


(12.82) 


where  the  tension  is  expressed  by 


T  =  vi<jc). 


(12.83) 


Fig.  12.5.  Probability  density  function  for  displacement  response; 

A*  =  0.5,  i  s  0.05,  e*  »  1.0,  o  digital  simulation  estimate. 
(From  Roberts.®®) 
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12.62  Stationary  Response  of  Oscillations  with  Nonlinear 
Dancing  to  Random  Excitation 

As  was  mentioned  at  the  beginning  of  this  section,  the  Fokker-Planck  equation  has 
been  solved  only  for  restricted  cases,  such  as  nonlinear  systems  with  nonlinear  restoring 
forces  or  systems  under  white  noise  excitatiott  Roberts^^  removed  these  restrictions  and 
studied  the  behavior  of  the  Fokker-Planck  equation  for  a  system  with  nonlinear  damping 
For  a  ncHilinear  system  with  nonlinear  damping,  the  equation  of  motion  is 


x+fiF{x,x)  +  Gix)  =  nit). 


(12.84) 


where  p  is  small,  F  is  an  odd  function  of  i;  (odd  as  was  used  by  Dalzell**  is  more 
convenient  to  manipulate),  and  n(t)  is  white  noise,  Gix)  being  the  restoring  term. 
The  two-dimensional  Fokker-Planck  equation  for 

pix,x)  =  pix,y\x(i,yo-,t) 


is  the  same  as  Eq.  12.30,  for  Eq.  12.20  in  the  form. 


dt 


I  d^p 
"^2 


(12.85) 


and  is  difTicult  to  solve  for  general  Fix,y),  so  Roberts  inverted  this  two-dimensional 
Fokker-Planck  equation  into  a  one-dimensional  Fokker-Planck  equation,  introducing  a 
physical  variable  called  the  energy  envelope  V(r) 

Vit)=^x^  +  Uix),  (12.86) 


therefore  x  =  j2Vit)-Uit)  where  Uix)  =  J  G(|)d^. 

0 

The  one-dimensional  Fokker-Planck  equation  for  p(v)  s  p(v|vo;  r)  is  in  the  form 

I  q2 

p  +2 

in  Eq.  12.22  or  Eq.  12.29,  and 

B(v),  C(v)  are  as  in  Eqs.  12.75-12.77 

l 


^__a_ 

dt  av 


PBiv)--\ 


W 

where  /  is  the  strength  of  excitation  as  /  * 
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C(v)  = 


1 


I  /iv-< 


-JT-——  I  \\y-U(x)dx 

lA(y) 


A(v) 


=  -f  — 
2} 


R 

dx 


U{x) 


The  integratioQ  range  R  is  such  that  [/(x)  <  v.  The  general  stationary  solution  is  given  as 
p(v)  =^lun  p(vlvo;r)  «  M(v)exp|- 


Yi2(v)|h 


(12.88) 


where 


e(v)=/^<« 

0 


(12.89) 


and  /:  is  a  normalizing  constant  Then,  from  the  relation  of  v,  x,  y(=  x)  and  reverting  to 
the  original  x,y  variables. 


p(x,  y)  =  p(x, ylxd,  yo;  0  =  C  exp< 


-(f)' 


jy^-t-U(x) 


y.  (12.90) 


When  the  equation  of  motion  is  expressed  as 

x  +  2^oevox(l+elH’')-hCD^x  =  /i(f),  (12.91) 

now  0)0  being  the  natural  frequency  of  the  linear  (e  =  0)  system,  Eq.  12.90  becomes 


1 

n  +  2-^ 

p(x,y)  =  C  po(x,y)cxp^ 

-a„e* 

„ 

2 

> 

<< 

where 
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e*=  ecDQOQ, 


P(i{x,y)  =  -:^exp|  +  y*) 


231 


(12.92) 


X  = 


x{t) 

ao  ’  ^ 


x{t) 

(OoOo 


1/2 


ao  is  the  standard  deviation  of  response  when  e  =  0,  e  *  =  e/a>o  is  the  nondimensional 
nonlinearity  parameter,  and  Po(x,y)  is  the  result  when  e*  =  0  (C  =  1  when  e  *  =  0). 


From  this  result,  the  statistical  moments  of  the  response  can  be  calculated.  For 
example,  Roberts  obtained 


Jn  e^(\-erf  0)  ^ -6 


(12.93) 


e  = 


1 

(3f*)l/2  • 


He  calculated  the  equivalent  linear  damping,  as  discussed  in  Section  1 0. 1 ,  using  the 
general  expression  Eq.  12.91  and  calculated 


a^~  6e* 


(12.94) 


for  n  =  2  in  Eq.  12.91,  which  is  a  closer  approximation  ofa^  than  Eq.  10.17.  He  also 
calculated  the  corresponding  perturbation  solution  as 

— ==l-3e*.  (12.95) 

^0 


This  is  slightly  different  from  Eq.  10.50,  which  was  shown  for  small  e.  All  three  results 
are  compared  in  Fig.  12.6.  Robert  calls  his  method,  which  introduced  the  energy  envelope 
V(r)  as  12.86,  the  Markov  envelope  theory,  and  indicated  in  Fig.  12.6  as  ME  theory. 


£• 

Fig.  12.6.  Variation  of  the  mean  square  of  the  response  with  e*. 
Simulation  results:  •  5o  =  0.05,  +  =  0.50. 

(From  Roberts.®®) 


The  equivalent  linearization  theory  is  abbreviated  as  EL  theory.  In  this  figure,  only  the 
perturbation  theory  appears  different  from  the  rest,  but  here  we  have  to  remember  that  in 
the  perturbation  method  this  is  the  first  order  approximation  and  we  can  improve  the  ap¬ 
proximation  by  increasing  the  order  of  the  approximation,  as  indicated  in  the  discussion 
in  Section  10.2.2. 

12.63  Nonlinear  Oscillation  in  Nonwhite  Excitation 

Roberts®'  further  extended  the  scope  of  applicability  of  this  method  -  d  solved  for 
nonlinear  rolling  in  nonwhite  excitation.  The  equation  of  motion  is  now 

Itp+^dtp)  +  k{<p)  =  M{t)  (12.96) 

or 

<i>'  +  ^Fi<p)  +  Gi(p)  =  x(t)  (12.96’) 

where  jc(r)  is  a  nonwhite  excitation,  and  can  have  a  colored  spiectrum. 

Again,  he  adopted  the  total  energy  envelope  as  Eq.  12.86 


where 


0 

=  j  G{^)d^ 
0 


and  considered  that  this  was  slowly  varying. 
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He  set 

(ip-  , 

^  =  -Vs\r?d 
2 

t/(0)  =  vcos^e 

introducing  the  phase  6  as  shown  in  Fig.  12.7.  After  manipulation  of  the  relations,  the 
equation  of  motion,  Eq.  12.96’,  was  inverted  into  the  equation  of  V  and^.  When  the 
phase  process  6  was  modified  into  a  process  A(r),  and  the  joint  process  Z  =  (V,  A)  was 
made  to  converge  into  a  Markovian  process,  the  transitional  distribution  function 
p(ZlZo;r)  was  found  to  be  governed  by  a  Fokker-Planck  equation  of  second  order.  The 
stationary  solution  of  the  Fokker-Planck  equation  p(Z)  was  obtained,  and  from  this 
expression  V  and  A  were  foimd  to  be  uncoupled,  and  p(V)  was  calculated.  With  the 
relations  of  V,  (f>  and^,  p(V)  was  modified  into  pi<l>,<p). 


or 


=  -  /2  Vi  sin  6 


t/(0)2  =  Vicosd 


(12.97) 


Rg.  12.7.  U,  Vvs.e,4>. 

(From  Roberts.®’) 

For  nondimensionalized,  nonlinear  rolling  expressed  as 

^  +  cup  +  b\^\  ^  ^  -  (f>^  =  x{t),  (12.98) 

Roberts  calculated  the  probability  density  function  of  the  nondimensionalized  amplitude 
of  rolling  A  by  the  ME  theory  and  compared  it  with  the  Rayleigh  distributirm  obtained  by 
the  linear  theory  as  in  Fig.  12.8.  also  compared  his  results  with  the  nonlinear  sim¬ 

ulated  data  obtained  by  J.F.  Dalzell®^  to  show  the  validity  of  his  method.  Examples  are 
shown  in  Figs.  12.9  through  12.11  for  a  variety  of  damping  coefficients  a  and  b.  where 
Q  =  CDp/too,  (Op  is  the  peak  frequency  of  the  excitation,  andOtv  is  standard  deviation 

of  the  input  process  x(t).  Process  3  in  the  figures  is  a  wide  banded  excitaticm  for  this 
example.  He  showed  many  other  results  of  comparison  for  other  types  of  excitation  with 
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different  bandwidths,  named  Process  1  and  Process  2,  used  in  the  simulation  by  J.F. 
Dalzell.*^  The  deviation  from  the  Rayleigh  distribution  in  Figs.  12.8  to  12.11,  and  the 
relation  of  the  standard  deviation  of  roll  Or  to  that  of  input  Ow  in  Fig.  12.12  shows  the 
extent  of  nonlinearity.  The  applicability  of  the  method  was  also  discussed. 


Rg.  12.8.  Probability  density  function  for  amplitude  A: 
a  =  0,  b  *  1,  =  0.036,  Q  =  0.90, 

Process  3.  (From  Roberts.®'') 


Rg.  12.9.  Cumulative  probability  distribution  for  amplitude  A: 
a  =  0,  b  =  1,  <7w  =  0.036,  £2  =  0.90, 

Process  3.  (From  Roberts.®'') 
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Fig.  12.11.  Cumulative  probability  distribution  for  amplitude  A: 

a  -  0.03,  -  0.036,  £2  *=  0.90,  b  =  0.1  and  3.0; 

Process  3.  (From  Roberts.®^) 


0^ 

0.15 

Or 

0.10 

0.05 


0 

Fig.  12.12.  Variation  of  standard  deviation  of  roll  o«  with  standard  deviation  of  wave 
input  a..:  a  =  0.01;  Q  =  0.90;  b  =  0.1. 0.3,  and  1.0;  Process  3. 

(From  Roberts.®"') 

12.7  PROBABILITY  DENSITY  FUNCTIONS  OF  AMPLITUDES,  EXTREME 
VALUES  IN  RELATION  WITH  THE  FUNCTIONAL  POLYNOMIALS 

12.7.1  Narrow-Banded  Case 

As  was  mentioned  in  Section  11.1,  when  a  nonlinear  response  r(r)  was  expanded  by 
the  Voltera  expansions  (or  functional  expansions) 

8n(ri,T2  .  .  .  .  T„)  X 


.  .  .  .  x(t-r„)  dti  dt2  .  .  .  .  dtn,  (12.99) 


the  terms  for  n>2  can  be  considered  as  the  modifying  terms  of  the  Taylor  expansion  of 
this  process  around  its  linear  term  for  r=1.  If  we  take  until  n=2,  and  using  small  e,  some¬ 
times  the  response  z(r)  is  expressed  as 


z(t) 


00 

=  / 


00  00 

11 


gl{x)  xit-x)  <k  +  €  g2{T:i,t2)x{.t-Xx)x{t-X2)  dXidX2.  (12.100) 


The  derivatives  in  terms  of  time  is 

00  00  00 


z(t)=  j  gl(x)  Mf-r)  dx-t2€  j  j  g2(TuX2)xit-Ti)xit-X2)  dXidX2,  (12.101) 
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thinking  of  the  synunetry  character  of  52(^1.  tj)  =  g2(T2.Ti).  On  the  basis  of  this  expres¬ 
sion  T.  Vinje®’  formulated  a  general  method  for  getting  the  probability  distribution 
function  of  the  extreme  values,  under  the  assumption  that  the  response  z(r)  is  nanow 
banded  and  the  input  x(r)  Gaussian  with  variance  r.  His  method  is  also  based  on  the 
assumption  that  the  probability  distribution  function  of  extreme  values  can  be  obtained  by 

the  joint  distribution  function  p{z,z)  only. 

Now,  we  start  from  the  statistical  moment  generating  function  to  get  p(zi,Z2) 

[here  21  =  2(f),  22  =  2(r)  2],  that  is,  from  the  double  Fourier  transform  of  p{z\,  22), 

62)  =  £[exp(i(0i2i  6222)}] 


OD  OD 

=  // 


exp(ieizi  +  i6222)p{zi,  Z2)dzidz2.  (12.102) 


Then  expanding  exp  (1^121  16222)  into  a  series  gives 


exp  (iOiZi  + 16222)  =  1  +  X 


m,n=0 


m\n\ 


(ie{r{i62)\ 


(12.103) 


assuming  m,  n  are  positive  integers  whose  sum  is  greater  than  zero. 
Inserting  Eq.  12.103  into  Eq.  12.102  gives 

0(01,02)  =  1  +  Z-^(t0l)'"(t02)". 


Here 


T*  OD 

=  E[z^,z^  ”  /  1  P^^u^2)dzidz2. 


(12.104) 


(12.105) 


From  the  Fourier  inverse  transform  of  Eq.  12.102, 


00  00 

/^(^i,^2)  =  -^2^  I  I  0(0i,02)exp(-i(0i2i +  0222)1^0  A-  (12.106) 
.00  .00 


Putting  Eq.  12.104  in  Eq.  12.106 
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00  00 

p(^i, 22) = I  I  iradz)" exp(- KOizi + e222)]d6ide2. 

—  00  —  00 

(12.107) 

This  Eq.  12.107  shows  that  p{z\,Z2)  can  be  calculated  if  we  know  all  the  m/i  th  order 
moments  of  this  probability  function. 

On  the  other  hand,  by  the  definition  of  the  cumulant  and  the  cumulant 
generating  function  K(i6\,i62), 


kn 


hi 


K{id\,  iOi)  =  log 0(^1, 62)  =  1!2! 


=s 

mji 


m\n\ 


(/^ir(t^2)". 


(12.108) 


Therefore 


0(^1,  ^2)  =  exp{^(t0i,  102)}  =  exp 


=  1  +Tr1 


J^-^mraeiT 

^  mini 


+  -^{  ....  }3  +  ...  .  (12.109) 


When  we  insert  this^(0i,02)  “to  Eq.  12.106, 


00  00 

piZi,Z2)  =  ^^^2  II  exp{Ar(i0i,  102)  -  t(0iZi  +  02Z2)}rf0ir/02 


go  OD 

"(2^1  I 


exp 


X-77(i0i)'"(ie2)" 
^  mini 


(12.110) 


From  Eqs.  12  104  and  12.109,  k„^  and  fim.n  are  related  as  follows: 
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(order  in  e) 


^1 

...  (0) 

.  .  .  (l+e2) 

^3  =  i«03  -  3/^011^02  +  ^01 

.  .  .  (e  +€^) 

^10  =yMlO 

...(€) 

^11  =/tll-A*lQ|M01 

...  (0) 

^12  = 12 -  2^1^11  +  2^10^01 

.  .  .  (€  -t-e^) 

*20  =  /U20-/UiO 

.  .  .  (1+e^) 

*21  =  H’2\  -  ^lOf^n -f^0lU20  +  ^lottoi 

...  (0) 

*30  =  M30-  3;riQU20  +  ^10 

.  .  .  (e  +^) 

(12.111) 


Following  these  general  formulations  by  Vinje,  M  Hinoio*^  calculated  the  proba¬ 
bility  distribution  function  of  the  wave  height,  treating  the  wave  process  as  nonlinear,  as 
was  mentioned  in  Section  9.2.1,  and  wiU  be  summarized  as  follows. 

From  these  relations  with  the  order  of  was  obtained,  which  is  a 

smallness  parameter  that  appeared  in  Eq.  12.100  as  is  listed  in  Eq.  12.111.  Expanding 
Eq.  12.110  around  6  =  0  into  a  Taylor  series  and  truncating  at  0(e)  gives 


-^^2o(O)0i-y*O2(O)^2 

X  [^l  +  fcio(0)(ifli)  +  |  ^3(0)(«2)^ 

+  1  ^'i2(O)(i0i)(i02)^j 


(12.112) 


where  ^y(O)  s  kijiO)  =  - 

AftCT  manipulation. 
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PiZuZ2)  = 


27r  ^20^ 


exp  - 


t2  -2 


2^20  2/:y2 


-’1 


2i 


\  y^  /  6  ^20  y^  ^  I  y^  /  6  ^02  ^^02  I  ^^02 


^3 


iy3 


€ 

+  — 


^12 


2  k(yiJk20 


H  i n  I 


(12.113) 


Here  H„  (x)  is  a  Hermite  polynomial. 


B„  (x)  =  i  [  e~^ix  +  i  Jl  tTdt 

ylJt  J 

~Q0 


^0(x)=l 
B I  ix)  =  x 
B2{x)^^-1 
Bjix)  =  x^-3x 


lBn+i{x)=xH„{x)-nH,^i{x)  (12.114) 


convention  of  (0)  is  omitted. 

Under  the  assumption  that  zi(f)  is  narrow  banded,  the  number  of  zero-crossings  of 
an  output  z(r)  is  equal  to  the  niunber  of  maxima. 

Therefore,  the  expected  number  of  z-up  crossings  per  unit  time  is  given  as  already 
shown  in  Section  12.5,  Eq.  12.44, 


=  \]k 


2I  piZi,Z2)dZ2. 


•.00 


(12.115) 
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As  already  shown  in  Eq.  12. 1 1 1 ,  the  cumulants  are  calctilated  from  the  m,n  th 
ir<oment  of  response,  and  the  m^th  moment  can  be  expressed  by  using  the  frequency 
functions  and  the  input  spectrum  as  some  of  the  examples  that  were  shown  in  Section 

11.4.  With  the  order  of  each  term  up  to  0  (e)  takoi  into  account,  and  with  the  spectrum 
function,  which  is  a  real  function,  the  cumulants  can  be  written  as  follows: 


359 


OD 


^10  =  €hio~€  I  sio})  G2i(0,-(0)dQ} 


k20  =  /»20 


go 

=  I  |Gi(a>),'' 


da) 


OD 

^  ^  =  I  (oh{a))  |Gi((o)pi/cy 


*12  =  e*i2 


go  OD 

-1/ 


(cjia>2  + 2fy2)  ^(twi)i'((2>2) 


X  [Gi(-fl>iX?l(-fi»2)<^(^‘>  1.^02) 


+  Gi(— <UiXji(— fl>2)G2(coi,tt)2)]  da}\da)2 


*30  =  e*30  =  5(G)iK«02) 


X  [Gi{-a)i)Gi{-Q>2)G2id>uo>2) 


+  Gi(— (yi)Gi(— <U2)^(<wi*<W2)]  d(o\dw2. 


where  indicates  a  complex  conjugate, 

s{o))  is  the  two-sided  spectnim  of  input  x(r), 
G\(a>)  is  the  linear  frequency  response  function 


Gi(<y)=  J 


(12.119) 
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and  G2(a>i,a>2)  is  the  quadratic  response  function 


GtiwuOii) 


00  00 

I  I  g2(Xi,X2)e-^'^^-^*^-dxidr2. 

—  m  —00 


Through  these  manipulations,  Hineno  obtained  his  expression  for  the  probability 
distribution  function  as 


P{z)  =  expl  - 


2h2o 


1+6^ 


h\Q  ^  hyo 


^20  6/I20  \  ^20 


-3  + 


U2 


2/102^20 


(12.120) 


The  probability  distribution  density  function  piz)  and  the  expected  l/n  highest  value 


are  obtained  from  the  relations 


P(2)  =  ——P{z), 

dz 


(12.121) 


^1/"  ~  00 


I  z  piz)dz  » 

- =  n  \  z  piz)dz. 

I  nz)dz 


(12.122) 


If  we  use  Eq.  12.120  for  P(z)  and  consider  that  in  real  calculations,  the  smallness 
parameter e  is  absorbed  in  the  computation  of  G2(o>i,a>2)  and  does  not  appear  explicitly. 


piz)  =  exp 


f-— ) 

\  2/120/ 


^10 

^  ^30 

hi2 

1 

^  /120  2/I20 

2/l2o/k)2 

,  ^  — 

/J20 

^30  ^ 

hl2  \ 

^30 

2^20  2/120/102  / 

Ihlo 

J 

^  A.  ^ 

6h^ 


20 


(12.123) 


This  is  the  final  expression  of  the  probability  distribution  density  function  of  the  maxima. 
Using  this  expression,  M.  Hineno  calculated  the  probability  characteristics  of  the  maxima 
and  minima  of  the  waves  that  were  treated  as  nonlinear  with  the  quadratic  response 
function,  as  shown  in  Eq.  9.10  in  Section  9.2.1, 
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G2{(oi,-(02)  =  (to?  -tyi) 


with  the  linear  response  assumed  as  Gt(to)  =  1. 


rig.  12.13.  Probability  density  function  of  the  maxima  of  waves. 

(From  Hineno.®^) 

Figure  12.13  shows  the  probability  distribution  density  function  for  maxima  of  the 
waves  compared  with  the  experimental  data  by  a  certain  research  worker  for  model 
waves  with  a  wave  spectrum  that  is  almost  of  the  Pierson-Moskowitz  type.  In  Fig.  12.13, 
the  -f  marks  indicate  the  experimental  data  analyzed  as  nonlinear  waves  and  the  other 
marks  (A,  O)  are  experimental  data  analyzed  on  the  assumption  of  linear  waves,  the  solid 
line  being  the  theoretical  relations  as  linear  for  this  model  waves.  Results  computed  by 
Eq.l2  .123  are  shown  by  a  dotted  line,  and  the  agreement  with  -i-  signs  is  quite  good, 
especially  in  the  larger  amplitude  range,  which  is  important  practically. 

Figures  12.14, 12.1S,  and  12.16  illustrate  the  calculated  results  for  nonlinear  waves, 
using  a  modified  Pierson-Moskowitz  wave  spectrum  with  //1/3  =  11.6  m,  foi  =  16.1 

sec  as  the  input.  From  these  figures,  we  can  find  the  extent  of  nonlinearity  of  the  waves 
and  also  the  effects  of  nonlinearity  or  the  effect  of  the  distortion  of  the  wave  forms  on  the 
difference  of  the  probability  distribution  function  for  maxima  and  minima. 


Hineno**  also  calculated  the  relative  motioD  of  a  semi-submersible  m  this  kind  of 
nonlinear  wave,  assuming  a  linear  response  to  the  excitation  by  the  waves. 


z/v^hgo 


Pig.  12.14.  Cumulative  probability  distiibution  function  of  wave  amplitude. 

(From  Hineno.®^) 


z/ /hgo 


Fig.  12.15.  Probability  density  distribution  function  of  wave  amplitude. 

(From  Hineno.®^) 

12.72  Wide-Banded  Case 

J.  F.  Dalzell^  (1984)  extended  this  technique  further.  He  did  not  assume  narrow 
bandedness  of  the  response  and  did  not  truncate  the  functional  polynomials  at  n=2  but 
continued  to  n=3.  He  thus  formulated  the  technique  for  calculating  the  probability  distri¬ 
bution  function  of  extremes  of  the  nonlinear  responses  to  Gaussian  inputs.  Here  the 
characters  of  the  nonlinear  frequency  response  functions  up  to  degree  3  are  assumed  to  be 
known  frcnn  the  analysis  as  discussed  in  Section  11.5.  The  nonlinear  response  Y{t)  to 
Gaussian  input  X(r)  is  expressed  by 
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n 


Rg.  12.16.  Expected  1/n  highest  values  of  wave  amplitude. 
(From  Hineno.®^) 


(Limits  of  integrals  -  oo  —  +  c»  are  omitted  throughout  this  section.) 

As  was  assumed  in  Section  11 .5,  here  the  kernels  or  the  nth  degree  impulse 
response  functions  gn(ti,T2  .  .  .  tn)  are  real,  time  invariant,  completely  symmetrical  in 
the  variables  g„(Ti,T2  •  •  •  ■*■„)  =  g„(T2.T3  .  •  •  t„,ti)=  .  .  •  for  any  rearrangement  of  the 
variables  Xj,  and  sufficiently  smooth  and  integrable  so  that  there  exist  n-fold  Fourier 
transforms, 

gnir\,X2  .  .  .  I  1  ■  '  ■  1  ■  <On) 

n 


exp 


d(Oi(ko2  ■  .  .  du}„ 


(12.125) 
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G„(C0i,W2  .  .  .  Ct)n)  =  gniri,T2,  .  •  .  T„) 


n 


exp 


n 

.  7=1 


\dt1dr2  .  ■  .  dx„.  (12.126) 


The  function  G„(toi,<U2  .  .  .  <u„)  is  the  nth  degree  frequency  response  function  and 

is  also  symmetric  in  its  arguments  G„(<U|.a)2  •  -  •  fl>„)  =  G„(a)2,t£)i,  .  .  .  o}„)=  ...  for 
any  rearrangement  of  because  the  impulse  response  functions  are  real. 

=  1,0)2,  .  ■  .  (o„)  (12.127) 

here  the  denotes  the  complex  conjugate. 

In  Dalzell’s  paper, ^  the  spectrum  5xr(o>)  was  defined  a  little  differently  from  those 
used  by  this  author  in  Parts  I,  II,  and  in.  He  used  2;r  rimes  oms(Q}) , 

Sxj^O)  )  =  2jts(,(o),  (1 2. 1 28) 

and  also  took  the  one-sided  spectrum 

t/xx[|<u|]  =  25(a))=-Sxt(a)),  foro)  >  0.  (12.128’) 

n 

The  initially  assumed  functional  polynonrial  process  was  reformulated  as  the 
response  to  a  white  noise  excitation,  and  a  new  set  of  fi'equency  response  functions  was 
defined  which  contains  both  characteristics  of  the  original  frequency  response  to  the 
excitation  and  that  of  the  excitation  spectrum. 

The  two-sided  spectrum  of  white  noise  is 

S^<o)  =  1,  (12.129) 

and  the  autocorrelation  of  white  noise  is  a  delta  function 

=  ~  I  exp[MOT]da)  =  d(T).  (12.130) 

We  think  of  the  spectrum  filter  L(q))  that  expresses  the  spectrum  of  inputSxdo)) 
with  the  white  noise  as 

S«((o)  =  a2  1L(q))|2  (12.131) 
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L{o))  = 


(12.131’) 


The  linear,  quadratic,  and  cubic  frequency  response  functions  connecting  the  white 
noise  W(t)  with  the  output  Y(t)  will  be 

aJJiio))  =  axUo))G{(o) 

alH2((0i,(02)  =  a^Li(Oi)Ua)2)G2((Oi,a)2)  (12.132) 

a|//3((Ui,G>2i03)  =  alL(a)i)L{a)2)L{(03)G^{(0i,a)2,(03). 

Then  y(f)  can  be  related  to  the  white  noise  input  IVfr)  as 
Y(t)  =  /ti(Ti)l^(r-ri)t*-i 

+  <7^  1 1  ^2(^1,  T2)W(t  -  Xi)W(t  -  X2)dXidt2 


+  oi 


hiiXi ,  X2,  X3)W(t  -  Xi)W(t  -  X2)W{t  -  Xi)dxidx2dXi,  (12. 1 33) 


where 


hiixi)  =  [  //i(<y)exp[icori]d(y 

2jr  J 

h2iXi,X2)  -  a  H2i<0i,Q)2)cxp[ii(0iXi+(02X2)]d0}id(02 


(12.134) 


%(tl,T2,T3)=-^^^|  I  |//3(Ctfi,a>2,Q>3)exp[/(a>iTi  +  a>2T2  +  <y3^'3)]^*t>l<^0>2<*t>3- 


The  derivatives  of  the  output  are  then 
ht)  =  Oj,j  h'i(xi)W(t-Xi)dxi 

,X2)W(f-Xi)W(t-X2)dridr2 


h3<Ji,X2,X3)W(t-Xx)Wit-X2)Wit-X3)dCidX2dt3,  (12.135) 


where 
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hliri)  =  f  Hii(o)cxpiia}ri)iko 

27t  J 

//  //2(tWi,fl>2)exp[j(o;iTi  +  (O2X2y\d0iid0)2 


(12.136) 


^3(ti,T2,T3)  =  J  I  j  liiia)uC02,(O2)cxp[i(Q)iti+a)2T2+(Oir3,)]da}idaj2d(0^. 


The  tenns  H„(w)  are  defined  as 

//i(fo)  =  i(oHii(o) 

H2(<Oi,(02)  =  i(Q}i+(02)H2{(0i,Q)2) 

Hi(a}i,a)2,Q}i)  =  i{(0i  +  (O2  +  0}i)Hiia)u(O2,Q)i) 


(12.137) 


and 


m  =  a;cjh 


hi(Ti)W(t-zi)dzi 


+  oi||  h2ituT2)W(t-Ti)Wit-r2)dtidt2 


+  al 


III  *3(ri,T2, 


t3)Wit  -  ti)W(t  -  T2)W(t  -  ri)dxidx2dx^,  (12.138) 


where 


I  ^i(<u)exp(Moz'i)dc> 

Tjf  J 

^2(Xi,T2)  =  ^^"2  j  J ^2(<Wl.<W2)exp[l((UiTi  +  a>2T2)]<*yiiity2 

^3(Ti,T2.T3)  =  J  J  J  ti3{<0i,W2,(03)cxp[iiWiri  +  (02t2  +  <03r3y]dcOidO}2dQ)7,, 


(12.139) 


and  are  defined  as 
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H2(0i\.(O2)  =  -{0}i-^(t>29H2{(J)\,(O2)  (12.140) 

Hi  {(01,0)2,  a)i)  =  - ((01  +  0)2  +  (Oi)^Hi((o  1,0)2, 0)3). 

In  addition,  the  characteristics  of  the  products  of  white  noise  were  fully  utihzed  as 
follows: 

^i(f-‘(^iW(t~T2)  ■  ■  ■  W(f-r/^)  =  0  for  N  odd, 

W(t-Ti)W(t-T2)  =  S(Ti  -T2)  =  d  12, 

-ti)W(T-  T2)W(t  - T3)W(t  -  T4)  =  <5 12^34  +  613^24  +  ^  14^23- 
W(t  -  Ti)I^(r  -  r2)W(t  -  T3)l^(r  -  T4)t^(r  -  T5)W^(r  -  ts) 

=  <5i2<534^56  +  <512^35(546  +  <5i2<536^45 
+  ^13^24<556  +  ^  13<525^46  +  ^  13^26<545 
+  ^  14<523<556  +  ^  14^25<536  +  <5 14<526^35 
+  5i5<534^26  +  <5i5^32<546  +  <515^36^24 
+  ^16^34<552  +  ^16^35<542  +  ^  16^32^45. 

where  dy  =  (5(t,  -  xj). 

Eq.  12.141  is  the  special  case  of  Eqs.  11.68, 11.69,  and  11.70  in  Section  11.5. 

If  we  do  not  assume  narrow  handedness  of  the  output  spectrum,  the  expected 
number  of  maxima  of  the  response  Y  greater  than  Y  =  ^  per  unit  time  is  approximated  as 

00  0 

^1^=1  I  \Y\  p(Y,Q,Y)  dY  dY.  (12.142) 

I  -X 

Similarly,  the  expected  niunber  of  minima  of  Y  less  than  y  =  |  per  unit  time  is 
approximated  as 
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I  “ 


Nf=  11  \Y\  p(Y,0,  f)  dY  dY. 

—  CC  0 


(12.143) 


Then,  the  expected  number  of  maxima  regardless  of  magnitude  per  unit  time  will  be 


Nc 


00  0 

=11 


m  p{Y,Q,Y)  dY  dY. 


(12.144) 


Similarly  the  expected  number  of  minitna  per  unit  time  will  be 


00  00 

iv;  =  1 1  m  p(y,Q,Y)  dY  dY. 


(12.145) 


—  00  0 


Because  maxima  and  minima  are  paired  in  the  same  record  of  response  -  N* , 
from  Eqs.  12.142  and  12.144,  the  probability  that  a  maximum  will  be  less  than  |  is  ap¬ 
proximated 


Prob[Maximum  ^  =  1  - 


(12.146) 


Similarly, 


Prob[Minimum  ^  = 


(12.147) 


Th«i,  the  probability  densities  of  maxima  and  minima  are  obtained  by  differentiat¬ 
ing  Eqs.  12.146  and  12.147  with  respect  to  as 


P^(l)  = 


0 

j  \Y\  p(^,o,Y)  dir 


—  00 


(12.148) 
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-  log  (ity  (it)''  (it) 


(12.152) 


Therefore 


<f>(it,ii,it)  =  exp  [K(it,ii,it)]  (12.153) 

where  j,  k,  and  m  are  positive  integers  whose  sum  is  greater  than  zero.  Therefore,  the 
inverse  transfonn  of  Eq.  12.150,  from  Eqs.  12.150  and  12.152,  is 
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j  j  j +  jj  <p(it,it,ir  )  dt,dt,dt 

=  exp  [ArOr,K,jr)j 

~  (2^  /  /  [-/yr-/y^-J^V+ArOr,jr, /f  )]  dt,dt,dt 

”  (2^1  /  [(*100 -10  it  +  {k(,io~Y)  ii  +  {kQo\-^it 

-■j{*200^‘+*020'‘*+*002f'  +2^110^  + 2*101? r  +  2Aoii/r } 

+  y  (,-,y  (,/)*(irrl  (12.154) 

j\k\m\  J 


yibn 


Here  /,  *,  and  m  are  now  positive  integers  whose  sum  is  greater  than  2. 
The  relations  between  joint  moment  and  joint  cumulants  are 

*100  =/?ioo 
*0io  =  A?oio 
*001  =fM30\ 

=  variance  of  Y 
*020  =  variance  of  Y 
*002  =  variance  of  Y. 


(12.155) 


Nondimensionalizing  the  variables  as  follows 

z  =  (y-*ioo)*5S 

z  =  (y-*bio)*^ 
2  =  (i'-*ooi)*ora 


(12.156) 


and 
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+  X  {isf  (is)’"  ds  ds  ds.  (12.159) 

jkm 

Expanding  the  characteristics  of  the  moments  as  shown  in  Eq.  12.159  into  joint 
moments  and  modifying  them  into  joint  cumuiants,  DalzeU  derived  the  expression  for 
joint  cumuiants  until  the  fourth  degree  from  the  modified  frequency  response  character¬ 
istics  H\((o),  H2((0\P2),  //3(a) i,ct>2,tt>3),  and  functions  ofeu  by  the  same  type  of  style  as 

Eq.  12.119  for  the  case  of  p(z,  z).  Since  these  manipulations  require  many  transactions 
involving  the  higher  order  terms  in  the  expansions,  he  checked  the  order  of  magnitude  of 
the  functions,  suggested  the  order  to  truncate  the  approximations  and,  by  laborious 
manipulation  utilizing  Hennite  polynomial,  he  obtained  his  approximation. 

From  these  expressions,  arbitrarily  denoting  the  standardized  maxima  or  minima  of 
response  by  v  he  finally  derived  the  expression  for  p(v,  0,’z)  using  the  cumuiants  up  to 
the  fourth  order  and  then  the  probability  distribution  function  for  maxima  and  minima  by 

0 

p\y>)  =  f  fil  p(v,  0,  z )  dz 
A'  00  } 


0 


z  p{v,  0,z)  dz 


«>00 


(12.160) 
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OD 

p~(y)  =  I  Izl  p(v,  0,  z )  d'i 

N  CC  J 


00 

=  ■^1  iXv,o.i)<(f. 


(12.161) 


Finally  he  derived  the  expressions  for  his  first  and  second  approximations  pt(v), 

pT(v)  and  P2{v),  piiv)  as  follows.  First,  by  neglecting  the  term  higher  (than  3)  order 
joint  cumulants,  the  first  approximation  is. 


Pl(v)  = 


e 

(2jr)'/2 


+  (1  V  exp 


v(] 

♦ 

e 

(12.162) 


-(1  V  exp 


V(l-f  2)1/2 
€ 


(12.163) 

where  0(a)  is  the  Gaussian  cumulative  distribution  function  and  e  is  the  spectrum  band 
width  parameter. 

The  6  was  introduced  in  Eq.  12.162  and  Eq.  12.163  from  the  relations  as  follows  that 
comes  from  the  characters  of  joint  cumulants. 


-^101  = 


km 


=  (l-e2)V2 


(12.164) 


=  (12.165) 

It  is  interesting  to  find  that  this  fust  approximation,  Eq.  12.162,  is  just  the  same  as 
the  one  that  Cartwright  and  Longuet-Higgins’^  derived  as  shown  in  Eq.  12.49,  and  Fig. 
12.1  in  Section  12.5. 
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DalzeU's  final  expressions,  including  the  effect  of  higher  order  cumulants  (until  3) 
that  take  account  of  the  higher  order  nonlinear  response  until  the  third  order,  are 


/?2(v)  =  5(v,e) 


+  Jt(v,e)  expi 1  0 

- 1 

1 

►J 

_  1 

\  ^  / 

L  ^  J 

(12.166) 


P2(v)  =  S(v,e) 


€ 


-Riv,€)  (1 


-€ 


:)l/2 


expl  - 


v(l 

€ 


(12.167) 


Here 


S(v,e)  =  1  +  V 


Asood+e^-Se^)  .  Aoosd -e^ 


+ 


(l+e2)(l-e2)i/2-Ai2o 


+  v%oo, 


(12.168) 
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/?(v,e)  =  v  + 


^120 


>1201 


-  +  3A30U 


^201 


6^300 


+  v^Asoo, 


(12.169) 


where  e  is  the  so-called  baud  width  parameter,  as  Eq.  12.165. 

As  referred  to  in  Section  1 1.5,  item  6,  Dalzell*^  generated  the  simulation  data  of  the 
cubic  nonlinear  response  to  check  the  validity  of  his  tri-spectrum  analysis'^  for  the 
Gaussian  excitation  of  the  waves,  as  expressed  by  the  Moskowitz-Pierson  type  spectrum. 


[-1.25(5)“ 

exp  - - ^ -  (12.170) 

co^ 

where  Uxx(o>)  is  a  one-sided  spectrum,  al  is  the  spectrum  area  and  variance,  and  tu©  is  the 
modal  frequency. 

Using  the  same  spectrum  parameters  for  nonlinear  response  as  he  used  before’’  and 
varying  the  excitation  level  by  changing ct,  totTjf  =  0.125, 0.25, 0.50, 0.75,  and  1.0,  he 
obtained  results  for  simulated  data  averaging  10  samples  for  each  case. 

He  calculated  the  probability  distribution  of  maxima  and  minima  and  compared  his 
results  with  Eqs.  12.162, 12.163, 12.166,  and  12.167. 

Figure  12.17  shows  the  comparison  of  probability  distribution  densities  of  response 
maxima  and  minima  estimated  for  the  simulations  with  those  of  the  first  and  second 
approximations  from  Eqs.  12.162, 12.163, 12.166,  and  12.167.  Here  z  represents  the 
normalized  maxima  and  minima  and  was  equal  to  v-A]oo,  Aiq©  being  the  normalized 
mean  and  v  the  standardized  maxima  and  minima  as  appeared  in  Eqs.  12.162  through 
12.167. 

Figures  12.18  and  12.19  compare  cumulative  distributions  of  response  maxima  and 
minima  estimated  from  the  simulations  with  those  of  the  first  and  second  approximations. 
In  these  two  figures,  the  standard  deviation  of  input  O,  (“SIGMA”)  is  the  same,  and  the 
band  width  parameter, 6,  (“EPSILON”)  was  different.  Here  the  first  approximation  is  ex¬ 
pressed  as  a  straight  line  (the  scale  of  the  cumulative  probability  is  so  chosen)  and  is 
marked  as  the  fitted  distribution. 

These  figures  indicate  that  the  final  expression  for  the  second  approximation  gives 
excellent  results  that  check  the  simulated  results  very  well. 
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-  SECOND  A^mcUUMATION 

—  ncsr  *moxMATioN 

ooo  swuunoN  ;  stammio  oevmtion  or  smt  >  iu»o  ;  or  ouwr  •  tj/t 


—  SECOND  AmtoxMA'noN 
- niST  AmoXttWTKM 

eoe  SMiAATioN  :  standaro  oevunoN  or  imt  •  Ijooo  ;  or  outpv/t  >  U7s 


Fig.  12.17.  Comparison  of  densities  of  response  maxima  and  minima 
estimated  from  the  simulations,  with  those  of  the 
first  and  second  approximations. 

(From  Dalzell.®®) 
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F 


ihcoRcncAL  osDiisuTiON  or  umu* 

EPSILON-  .482  .  SIGMA-  1.00 


s 

5 

> 


o 


o 


TMEONCTKAL  OISTRainiON  OF  MAXIMA 
EPSILON-  .482  .  SIGMA-  1.00 


u 


Fig.  12.18.  Comparison  of  oimulative  distribution  of  response  maxima  euid 
minima  estimated  from  the  simulation,  with  those  of  the  first 
(fitted  distribution)  and  second  approximation  -  sample  1 . 
(From  Dalzell.®®) 
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UAXMA  .  STANOMO  OCVMnONS  .  INPI/To  1.000  ;  0inT>i;T-  I.J76 
A  738  POINT  sample  fitted  TO  THE 
THEORETICAL  DISTRIBUTION  OF  MAXIMA 
EPSILON-  .691  .  SICMA-  1.00 


MINIMA  .  STANDAPO  DEVIADONS  .  INPUT-  1.000  :  OUTPUT-  1.376 
A  735  POINT  SAAlPlE  FITTED  TO  THE 
theoretical  DISTRIBUTION  OF  MAXIMA 
EPSILON-  .691  .  SICMA-  1.00 


Fig.  12.19.  Comparison  of  cumulative  distribution  of  response  maxima  and 
minima  estimated  from  the  simulation,  with  those  of  the  first 
(fitted  distribution)  and  second  .approximation  -  sample  2. 
(From  Dalzelt.®®) 
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CHAPTER  13 


EXTENSION  OF  MODEL  FITTING  TECHNIQUES  TO 
NONLINEAR  PROCESS 


13.1  INTRODUCTION 

Suppose  we  have  a  discrete  random  process  X,  and  a  purely  random  process  e, ,  and 
X  is  expressed  by 

Xt  =  QA(Xt-i,Xt.2y '  'Xi^)  +  €t.  (13.1) 

When  Qyi  is  a  linear  function, 

Xt  =  Cl  X,_i  +  02  X,_2  +  •  •  •  +  a„  X,_„  +  €, ,  (13.2) 

X  is  called  as  an  a.itoregressive  process  (AR)  of  nth  order,  as  discussed  in  Part  H, 

Chapter  5. 

If  X|  is  expressed  as 

Xi  =  QM(€t-l  ,€i-2,€i-2,€t-m  )  +  ^i,  (13.3) 

and  Q„  is  a  linear  function, 

X,  =  bi€,.i  +  b2  €^2+-  ■  '  +  bn  +  (13.4) 

then  X,  is  a  moving  average  model  (MA)  of  mth  order,  also  discussed  in  detail  in  Part  n. 
Chapter  5. 

When  these  Q/i  or  Qm  functions  are  not  linear  functions  but,  for  example,  the  poly¬ 
nomials  of  Xf_„  OT€,_„ ,  then  X,  is  no  longer  a  linear  process  and  is  called  an  expanded 
AR  model  or  an  expanded  MA  model. 

A  Voltera  type  process,  as  we  saw  in  Section  11.1  in  Eq.  11.1,  is  this  expanded 
moving  average  process.  More  generally, 

Xt  =  QaO^i-x  Xt-2y  Xt-,^  -t-  ,  e,_2  )  +  e,  (13.5) 

is  an  expanded  ARMA  model  when  Q„  are  polynomials  of  variables. 

For  expanded  AR,  MA,  and  ARMA  models,  there  is  no  general  way  to  solve  the 
process.  When  the  process  is  expressed  as, 

Xt  +  a\  Xt-i  +  02  X,_2  •  •  '+Ok 

m  V 

=  €t+bi  €,_!  +  •  •  ■  +bi  €,_/+  ^  X  (13.6) 

»■=  ij  =  i 

Eq.  13.6  is  called  a  bilinear  model.  The  scope  of  nonlinear  models  that  have  been  analyti¬ 
cally  developed  is  rather  limited.  A  few  efforts  have  been  made  along  the  following 
mo^ls:  i)  a  simple  bilinear  model,  ii)  a  threshold  autoregressive  model,  and  iii)  an 
exponential  autoregressive  model. 


379 


13.2  BILINEAR  MODEL 


As  the  simplest  example  of  a  bilinear  model,  we  take  the  first  order  model 

X,^i  =  aXt  +  b€,+i  +  c€tX,  (13.7) 

Priestley,^  in  studying  the  bilinear  model,  assumed  that  Eq.  13.7  can  be  expanded  into 
the  form  of  Voltera  functions.  Eq.  1 1 .8  as, 

00  00  00 

X,=  2  X  X  S  U,V  Ul-U  f4-V  +  •  ■  •  •  (13.8) 

u=o  U^OV -o 

Here,  the  general  transfer  functions  are 

u 

00  00 

r2(a)i,tt>2)=  X  X  gu.v  (13.9) 

u  =  V  =  o 


Assuming  Eq.  13.7  is  expanded  into  the  form  of  Eq.  13.8,  he  derived  r](a)),  setting 
e,  =  e“"inEq.  13.7,  as 

ri(<0i)  =  — - .  (13.10) 


Similarly  setting  €,  ~  e‘“'‘  + 

r2(<Oi,m2)  =  -7- 


2  gi(wi-wj)_^j 

\cb 


(r(mi) + r(ct>2)} 


e^' 


Particularly  along  the  diagonal  a>i  =  o)2  =  o). 


T2i(O,0))  = 

More  generally,  Priestley  showed  that 


cbe^ 


r*(ty,tu,-  •  ty)  = 


c*  ^b  e“ 


(€' -a)  -  ■  a) 


(13.11) 


(13.12) 


(13.13) 


From  these  relations,  he  found  thatri(mi,a>2)  included  all  the  a,  b,  and  c  of 
Eq.  13.7.  Thus  he  says,  the  bilinear  process  expressed  by  Eq.  13.7  is  invertible  to  the 
functional  polynomial  model  of  Eq.  13.8. 

This  bilinear  model  can  also  be  approximated  he  says,  by  the  generalized  autore¬ 
gressive  model.  The  first  order  bilinear  model  can  be  expressed  by  setting  b  =  1  in 
Eq.  13.7  (without  losing  generality). 
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X,^l=aX,  +  C^,  €,+  €,^1.  (13.14) 

We  can  modify  this  equation  to 

jl  +cXt  B^€t^i=Xi^i-aX,.  (13.15) 

Here  B  is  the  backward  shift  operator.  For  small  c,  £q.  13.15  c«n  be  inverted  to 

=  X,^i-aX,~cX,^  +  ca  X,  X,.i.  (13.16) 


This  is  a  generalized  autoregressive  model.  If  the  product  term  X,  is  neglected,  initial 
estimates  of  a  and  c  can  be  obtained  by  a  least  squares  approach.  If  the  product  term  is 
not  neglected,  a  nonlinear  least  squares  approach  can  be  used  to  get  a  and  c.  After  initial 
estimates  oq  and  cq  are  obtained  €,  can  be  estimated  by 

e,vi  =X,+i-aX,-cX,  €,  for  r=  1,2,  •  •  ■  N.  (13.17) 

Recursively  starting  frome,  =  0,  and  using  {X/,e,|  for  r  =  1,  •  •  •  N,  new  estimates  for  a 
and  c  can  be  found  to  minimize 

5;|x,+i-ax,-cx;€,j.  (13.18) 

/=  1 

Until  the  estimates  converge,  this  procedure  is  repeated.  Subba  Rao*’  smdied  this 
bilinear  model  and  gave  several  examples. 

13.3  TORESHOLD  AUTOREGRESSIVE  MODEL 

The  threshold  autoregressive  model  introduced  by  Tong^  is  generally  expressed  as 
Xi  +  afX-i  +  •  •  •  +  afx,.i  =  (13.19) 

where are  constants  in  Region  for 

(X^i, •  •  •  X,.i)€ i=l,-  ■  k.  (13.20) 


Region  is  in  the  /-dimensicna.:  <2(  clidean  space  R(  For  example,  the  first  order 
threshold  autoregressive  model  TAR(l)  is 


(13.21) 


where  the  coefficient  a  differs  by  the  size  of  Ai_/.  Priestley^^  indicates  that,  if  we  consider 
a  bilinear  system  in  which  the  physical  input  Ui,  output  X,  and  noise  €;  are  related  by 


Xf^i  —  aXi^  cUf  Xi  + €  1+1 , 

'  and  if  is  determined  by  a  feedback  mechanism  of  the  form 


(13.22) 
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f  +a  if  X,  <  d 
I  -a  ifXr^d' 


then  the  model  X,  can  be  expressed  by  a  threshold  model 


y  _  f  Xi  +  €i^i .  for  X,  <  d 

I  X,  +  €t+i ,  for  X,  d 


(13.23) 


(13.24) 


where 


/  =  a  +  a  c 

I  -a-a  c. 


(13.25) 


This  model  gives  a  nonlinear  process  as  shown  by  Eq.  13.22.  It  can  be  called  a  piece- 
wise  linear  approximation.  As  will  be  shown  later  in  Section  13.5,  Ozaki^'  made  it  clear 
that  this  piece-wise  linearization  is  a  special  case  of  his  nonlinear  threshold  autoregres¬ 
sive  model. 

13.4  EXPONEhTTIAL  AUTOREGRESSIVE  MODEL 

The  nonlinear  random  vibration  system  of  one  degree  of  freedom  is  expressed  as 

m  -H  gi{X(f)}  •  m  -h  g2(X(r)}  X(r)  =  n(f)  (1 3.26) 

where  n{f)  is  a  random  noise  excitation. 

When 


(13.27) 

Eq.  13.26  becomes 

m  =  -g2M  X(r)-gi{X(r))  Y{t).  (13.28) 

From  these  two  equations  and  the  state  space  expression  as  explained  in  Section  12.3 


m 

1 - 

o 

Xit) 

o 

1 _ 

m 

-«2{^(f))  -glM 

1 

+ 

1 

Si 

j 

or,  in  vector  notation. 


V(t)=f[x(t)\vit)  +  nit), 

where 


V(r)  = 


Xit) 

Xit) 

Yit) 

Xit) 

(13.30) 


(13.31) 


So,  finding  the  appropriate  expression  forgi(-),  g2{-)  in  Eq.  13.28  is  the  same  as  finding 
the  appropriate  function  /{Ar(f)j  for  Eq.  13.30.  When  these  two  functions  gi(-),  g2(-)  are  in 
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AMPLITUDE 


the  special  fonns  as  follows,  the  system  is  called  a  Duffmg  type  or  a  Van  del  Pol  type 
oscillator,  respectively,  as  describ^  in  Chapter  10. 

Duffing  type: 

g2[X{.t)\  =  a+^X^  (13.32) 

Van  del  Pol  type: 

gi{x(f)}=-cji-:s((/)2] 

g2(X(r)}  =  a.  (13.33) 

For  the  Duffing  type  oscillator,  under  harmonic  excitement  F  cos  cot. 


X(t)  +  cX  +  oY  +  ^X^  =  F  cos  (Ot. 


(13.34) 


We  know  that,  with  harmonic  excitement,  the  response  shows  damping  phenomena, 
as  indicated  in  Fig.  13.1,  and  in  this  case  the  natural  frequency  is  dependent  on  amplimde. 
So  we  can  call  this  oscillation  an  amplitude-dependent  period  shifting  oscillation. 


HARD  SPRING  TYPE 


SOFT  SPRING  TYPE 


Hg.  13.1.  Duffing  type  oscillator. 
For  the  Van  del  Pol  oscillator. 


X-c{l-X2jx  +  aX  =  0,  forc>0,a>0 


(13.35) 


the  system  possesses  a  limit  cycle,  because  when  the  amplitude  X  is  small,  the  damping 
becomes  negative  and  X  starts  to  diverge,  and  when  X(t)  becomes  large,  the  damping  be¬ 
comes  positive  and  the  amplitude  starts  to  decay.  This  system  remains  in  oscillation 
without  any  excitation. 

When  we  have  random  noise  excitation  n(r) 
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(13.36) 


X-cil-X^)X  +  aX  =  n{t), 

the  system  will  produce  a  perturbed  limit  cycle. 

and  Haggan  and  Ozaki’^-’*  proposed  a  new  type  of  nonlinear  model 
called  an  exponential  autoregressive  model  through  the  following  considerations  In  Eq. 
13.26,  when  gi  and  g2  are  constants,  the  system  becomes  a  purely  linear  oscillator  and  the 
equation  of  motion  is  expressed  as 

X  +  cX  +  aX  =  n{t).  (13.37) 

This  expression  can  be  inverted  into  an  ARMA(2.1)  model,  as  was  explained  in  some 
detail  in  Section  6.3.2  of  Part  n,  as 


X,  =  +  02-y/-2  +  ^if/-i  +  •  (13.38) 

Here  n(f)  is  a  continuous  Gaussian  white  noise  and  6/  is  a  discrete  Gaussian  white  noise. 
The  oscillation  expressed  by  Eq.  13.37  is  governed  by  its  characteristic  equation,  as  dis¬ 
cussed  in  Section  6.3.2, 

fi^-¥Cfi+a  =  0  (13.39) 

and  by  its  roots  (or  eigenvalues) 

±  iyla-{c^/4).  (13.40) 

We  know  this  model  A)  diverges  when  c  <  0  and  converges  when  c>  0.  On  the  other 
hand,  the  model  expressed  by  Eq.  13.38  diverges  when  the  roots  of  its  characteristic 
equation 

2^-012-02  =  0  (13.41) 

are  outside  the  unit  circle  and  converges  to  a  stable  process  when  two  roots  of  Eq.  1 3.41 
are  inside  the  unit  circle,  as  described  in  some  detail  in  Sections  5.2.4  and  5.2.3  and  in 
Fig.  5.16. 

From  Eq.  13.41,  when  0  J  ^  402,  i.e.,  when  the  roots  are  unequal  and  complex 
(conjugate  to  each  other)  zone  [  n  ]  in  Fig.  5.16, 

Z  =  ^  ±  4  iV-402-0f,  (13.42) 

since  IzP  =  02,  when -02  >  1  or02  <  - 1,  this  model  diverges  and  when  1  >  -02  >  0  or 

0  >  02  >  -  1,  this  model  becomes  a  stable  process. 

From  the  discussion  of  Green’s  functions  of  ARMA(2.1)  in  Section  5.2.4  or  of  the 
autocorrelation  of  AR(2)  in  Section  5.2.3,  we  know  that  damping  is  determined  by 

7-02,  i.e.,  by  02.  and  the  frequency  is  dependent  on0i. 

In  the  analysis  of  a  ship’s  nonlinear  rolling  that  comes  from  the  nonlinear  restoring 
moment  (the  Duffmg  type),  Ozaki  and  Oda^  first  fitted  a  nonlinear  model  of  the  type 

X,  =  01  X,_i  -t-  02  X,_2  +  n  Xf_i  -I-  e, .  (13.43) 
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(13.43’) 


Equation  13.43  can  be  transfonned  into 

=  (^1  +  TT  ^»-l  +  4^2  ■X/-2  +  f  f  ■ 

This  expression  shows  that  the  coefficient  of  the  first  autoregressive  coefficient, 
changed  ffom0)  in  Eq.  13.38  to (^i  +  jr  in  Eq.  13.43’.  This  means  the  frequency, 
determined  by  the  coefficient  of  X,_i,  became  amplitude  dependent.  Although  this  looks 
fine,  Ozaki  and  Oda  found  that  the  time  series  X),  defined  by  the  nonlinear  autoregressive 
model  Eq.  13.4’  was  not  stable  but  could  be  diverged.  Therefore,  instead  of  Eq.  13.43, 
they  proposed 

Xt  =  i4>i  +  Xf^i  +  <f>2  X^2  +  f t  •  (13.44) 

This  is  the  exponential  type  of  autoregressive  model  for  a  Duffing  type  oscillator.  By  ex¬ 
pressing  it  this  way,  we  can  make  both  roots  A(0),  A(0)  of  the  instantaneous  characteristic 
equation,  when  X^.i  =  0, 

A2-(^j+;r,)-^2  =  0  (13.45) 

and  also  the  roots  A(  <» ),  A( « )  of  the  instantaneous  characteristic  equation  when 

X^,  -*  00 

JL^-4>iZ-4>2  =  0  (13.46) 

lie  inside  the  unit  circle  as  shown  in  Fig.  13.2.  Accordingly,  with  Eq.  13.44  as  the  model, 
we  can  express  the  amplitude-dependent  and  stable  oscillation  under  white  noise  oscilla¬ 
tion  excitation. 


Fig.  13.2.  A(0),  A(  00 )  for  stable  Duffing  type  model. 

When  ni  >  0,  the  model  corresponds  to  a  hard  spring  system  and  when  <  0  the 
model  corresponds  to  a  soft  spring  type  oscillator.  Haggan  and  Ozaki  (1979)^  and 
Ozaki^^"^  showed  an  example  of  this  type  as 


.00  0.00  4.00 


(13.47) 


X,  =  (1 .5  +  0.28  e-^h  Xt_i  -  0.96  X,_2  +  e, 
where  a}  =  0.025,  namely  e,  is  N[0, 0.025]. 

The  eigenvalue  A  stays  lAl^  =  0.96  for  25®  <  Arg.  (A)  <  40°,  and  actually 
moves  between  Ao  =  0.89  ±  0.411  and  A®  =  0.75  ±  0.63)  when  IAi_il  changes  between 
0  and  *  as  schematically  shown  in  Fig.  13.2.  The  generated  time  history  is  shown  over 
r  =  1-100  in  Fig.  13.3.  To  check  the  characteristics  of  this  model  expressed  by  Eq.  13.47, 
the  model 

X,  =  (1.5  +  0.28^-^'-')  Xm-0  96  X,_2  +  a,  (13.48) 

was  simulated, where  d,  is  no  longer  random  but  is  the  form  a,  =  sin{2n^  f{t)  ■  rj,  where 
the  frequency /(r)  changes  with  time. 
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Rg.  13.3.  Generated  exponential  AR  model 

X,  =  (1.5  +  0.28er^M )  Xm  -  0.96X«  +  e  „  e , :  N[0,  0.025] . 

(From  Ozaki.®®) 

Figures  13.4  and  13.5  show  these  simulations.  In  Fig.  13.4,  the  frequency  y(r) 
increases  with  time  from  0.005  to  0.1,  and  in  Fig.  13.5  the  frequency /(r)  decreases  with 
time  from  0.1  to  0.005.  In  Fig.  13.4,  the  amplitude  suddenly  l^omes  small  at  aroimd 
/=  0.062;  in  Fig.  13.5,  the  amplitude  changes  abruptly  at  around/=  0.052  and  actually 
demonstrates  the  jump  phenomenon  shown  in  Fig.  13.1. 

In  the  same  way,  by  making  the  damping  amplitude  dependent,  02aki,®^’^  proposed 
the  model  for  expressing  the  Van  der  Pol  type  oscillator  as 

Xt  =  <f>iXt-\  +  {<f>2  +  XiJ2  +  f  1  (13.49) 

or,  more  generally, 

X,  =  {ip\-\-7ti  X,_i  +  (02  +  7r2e“^*')  X,_2  +  ei.  (13.50) 
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INPUT  a, 


Fig.  1 3.4.  X,  =  (1 .5  +  Q.28e-^h  -  0.96X^  +  a , . 

Input  a,  >  sinjsinTr  flO '  ^0  >s  increasing  by  time. 

(From  Ozaki.®®) 

In  Eq.  13.50,  the  coefficients  are  to  be  chosen  to  make  the  damping 

coefficient  negative,  when  the  amplitude  X,_i  is  small  and  the  oscillation  starts  to  diverge, 
and  to  make  the  damping  coefficient  positive,  when  the  amplitude  is  large  and  the 
oscillation  starts  to  decay.  Mathematically 

A2-(^i  +  ;ri)^-(02  +  ^2)  =  O,  (13.51) 

has  roots  outside  the  unit  circle,  and 

A2-0iA-02  =  O  (13.52) 

has  roots  inside  the  unit  circle,  as  shown  schematically  in  Fig.  13.6 


387 


00  0  Ol  O-*^  OO  t'  00  0  00>-' 


Fig.  13.5.  X,  =  (1.5  +  0.28e"^w)  -  0.96X^.s  +  a, . 

Input  a,  =  sin|2;r  HJ)  ■  /(/)  is  decreasing  by  time. 

(From  Ozaki.®®). 

As  an  example,  Ozaki  showed  that,  for  a  model 

X,  =  (1.95  +  0.23  e-^h  X^i  -  (0.96  +  0.24  X^2  +  e, ,  (13.53) 

the  characteristic  roots  are  Xo  -  1.09  ±  0.109/  and  =0.975  ±  0.0968/,  and  IX)^ 

actually  moves  between  UqI*  =  1.20  to  U«l*  =  0.96,  which  shows  Van  del  Pol  type 
oscillation,  as  shown  schematically  in  Fig.  13.6.  Figure  13.7  shows  that  the  model 
without  €( ,  which  starts  with  different  initial  values,  approaches  to  the  same  limit  period. 
Thus  this  shows  that  the  model  expressed  by  Eq.  13.53  has  a  stable  limit  period. 


Fig.  13.6.  2(0), >l(®)  for  Van  del  Pol  type  model. 
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Fig.  13.7.  X,  =  (1 .95  +  0.23e'^')Xw  -  (0.96  +  0.24e"^i)X^^,  with  different 
initial  values  (without  white  noise). 

(From  Ozaki.®®) 

More  generally,  Ozaki®*  proposed  the  higher  type  Duffing  and  Van  del  Pol  nonlinear 
model  as 


Xt  -  gl(Xt-i)  X|_1  +  g2(^f-l)  f . 

(13.54) 

where 

gi(Xf_i)  =  +  7t2^Xj_i  +  • 

(13.55) 

giiX^i)  =  ^2  +  (jio  ^  +  nf^Xli  +  • 

(13.56) 

The  term  g(-)  is  expressed  by  a  constant  plus  a  Hermite  type  polynomial.  When 
ri  =  0,  r2  =  0  in  these  equations,  an  equation  of  the  type  of  Eq.  13.50  is  obtained. 

To  determine  the  order,  Akaike’s  criterion  (AIC)  was  shown  to  be  applicable  for 
these  nonlinear  models,  too. 


13.5  NONLINEAR  THRESHOLD  AUTOREGRESSIVE  MODEL 

Ozaki^  extended  and  generalized  his  model  further  and  proposed  his  nonlinear 
threshold  autoregressive  model  and  a  unified  explanation  of  the  nonlinear  models. 

For  example,  the  exponential  autoregressive  model  that  he  used  as  an  example  of  a 
Van  del  Pol  type  nonlinear  process,  Eq.  13.53, 

X,  =  (1.9.^  +  0.23  X,_i  -(0.96  +  0.24 )  X,_2  +  e, 

was  shown  to  be  approximated  by  Tong’s  linear  threshold  autoregressive  model,  as 


1.95  X,_i-0.96  X,_2  +  e, 
2.18  X,_i- 1.20  X,_2  +  e, 


>0.5 

iflX,_il<0.t. 


(13.57) 
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The  characteristic  roots  of  the  model  jump  from  Ao[  W,oI‘  ®  ]  to 

A«[  IA«I*  =  0.96j  or  vice  versa  as  in  Fig.  13.6  and  also  shown  by  a  step  function  in 

Fig.  13.8.  While  in  the  model  as  is  Eq.  13.53,  the  path  of  two  rootsAo  andA*  and  A  at 
arbitrary  Xt^i  is  specified  by  a  continuous  Hermite-type  polynomials  in  the  equation,  as  is 
shown  by  a  continuous  curve  in  Fig.  13.8. 


Xm 


Rg.  13.8.  Path  of  characteristic  roots  of  threshold  model  and 
exponential  AR  model. 

(From  Ozaki.®®) 

Ozaki  argued  that  the  threshold  linear  AR  model,  however,  could  not  be  expected  to 
give  a  good  enough  approximation  to  nonlinear  vibration,  and  appropriate  nonlinear 
threshold  AR  model  must  be  formulated.  His  point  is,  we  have  to  consider  the  stepwise 
dynamics  of  restoring  and  damping  force,  but  at  the  same  time,  the  orbitally  stableness 
and  independence  of  the  limit  cycle  of  the  Van  del  Pol  equation  must  be  maintained. 

Instead  of  the  linear  step  function  approximation,  nonlinear  approximation 
considered  was  as  follows. 

^  ^  r  1-95  X,_i-0.96  X^.2  +  e, ,  for  IXmI  ^  10 
'  ■  (2.18-0.23  Xli)  (1.2 -0.24  X^)  X,.2  +  e,, 

for  IXmI  <  1.0.  (13.58) 

Equation  13.58  can  give  the  characteristic  roots  that  move  much  more  smoothly  than 
those  of  Eq.  13.57.  Thus,  the  general  form  of  the  nonlinear  threshold  autoregressive  mod¬ 
el  for  nonlinear  vibrations  was  proposed  as, 

j  for  IAr(_il  ^  T  (13  59) 

1  fl(Xi-i)  Xf^i  -(-•••  ■\-jp(Xt-p)  Xt-p  +  €, ,  for  lX,_il  <  T 

where 

f^iX)  =  -t-  nfx  +  •  •  •  -I-  X"'. 

Schematically,  the  behavior  of  the  characteristic  roots  for  (1)  linear  threshold  model, 
(2)  nonlinear  threshold  model,  and  (3)  exponential  AR  model  are  expressed  as  shown  in 
Fig.  13.9. 
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Fig.  13.9.  Schematical  expression  of  behaviors  of  characteristic  roots 
by  Ozaki.®® 


Ozaki***  investigated  the  stability  and  existence  of  limit  cycles  and  their  relation  to  the 
form  of  the  characteristic  equation.  For  example,  he  showed  that  a  nonlinear  threshold 
model. 


y  ^  f  0.8  X,_i+€,,  for  lX,_il  Sl.O 

'  I  (0.8  +  1.3  X^_i  -  1.3Xti)  X,_i  +  e, ,  for  lX,_il  <  1.0  (13.60) 

has  characteristic  roots  that  behave  as  shown  in  Fig.  13.10.  This  model  also  has  three 
stable  singular  points  and  gives  a  process  that  fluctuates  around  one  of  these  points  and 
jumps  from  one  stable  singular  point  to  another,  depending  on  the  white  input,  as  shown 
in  Fig.  13.11. 

With  these  discussions  and  examples,  Ozaki  showed  the  greater  generality  of  his 
exponential  models. 


Fig.  13.10.  Behavior  of  characteristic  roots  of  a 
nonlineeir  threshold  AR  model. 
(From  Ozaki.®®) 
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Rg.  1 3,11 .  Time  history  of  a  nonlinear  threshold  model  that  has 
characteristic  roots  that  behave  as  Fig.  1 3.1 0. 

(From  Ozaki.®®) 
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CHAPTER  14 


CONCLUSIONS  FOR  PART  HI 

The  so-called  “spectrum  correlation  method”  discussed  in  Parts  I  and  II  has  been 
shown  to  be  a  powerful  way  in  the  analysis  of  stochastic  processes,  but  only  when  the 
processes  are  linear.  Many  steps  in  the  manipulations  in  Parts  I  and  n  are  based  on  the 
assumption  of  linearity  of  the  process. 

In  the  analysis  of  seakeeping  data,  as  is  usual  in  the  general  engineering  field,  many 
phenomena  can  be  approximated  as  linear,  and  so  spectrum  or  correlation  analysis  has 
played  a  significant  role  in  advancing  the  technique  of  handling  those  data.  Now, 
however,  the  nonlinearity  of  seakeeping  data  for  ocean  vehicles  and  structures  has 
gradually  been  introduce,  as  mentioned  in  the  Introduction. 

Here  in  Part  HI  of  this  lecture,  methods  for  treating  the  nonlinearity  in  the  stochastic 
process  analysis  were  summarized  and  reviewed.  The  conclusions  obtained  here  are  as 
follows; 

1.  Several  works  on  the  effect  of  the  nonlinearity  of  ocean  waves  on  its  spectrum 
were  reviewed.  It  is  now  clear  that,  if  necessary,  we  can  get  the  effect  of  nonlinearity  on 
its  spectrum  as  well  as  on  the  probability  distribution  of  the  maxima  and  the  minima  of  its 
amplitudes.  It  is  also  clear  that  the  noiuinearity  of  ocean  waves  is  ustiaUy  quite  small. 

2.  Even  when  the  norlinearity  of  the  waves  is  small,  the  response  of  ocean  vehicles 
and  structures  might  be  nonlinear,  because  of  the  very  low  frequency  characteristics  of 
their  responses  that  respond  rather  severely  to  the  higher  order  nonlinear  excitation  by 
waves  or  because  of  the  nonlinearity  in  their  response  characteristics.  For  these  cases, 
when  the  nonlinearity  is  weak  and  the  response  characteristics  are  expressed  by  weakly 
nonlinear  equations  of  motions,  the  equivalent  linearization  and  perturbation  methods  can 
be  applied  if  the  excitation  is  approximately  linear.  The  perturbation  method  was  applied 
by  this  author  rather  early  to  ship’s  oscillation,  and  now  both  methods  are  well  formulated 
as  shown  in  Chapter  10.  These  methods  are  applicable  to  obtaining  the  frrst  approxima¬ 
tion  of  weakly  nonlinear  damping  and  restoring  oscillations  under  random  excitation.  To 
proceed  to  the  second  or  third  order  approximation  is,  however,  not  necessarily  easy. 

3.  The  Voltera  or  functional  expansion  is  one  appropriate  way  to  express  the  weakly 
nonlinear  response  to  random  excitation,  and  the  polyspectral  analysis  is  a  reasonable 
way  to  get  the  higher  order  nonlinear  frequency  characteristics.  The  procedure  for  this 
analysis  was  summarized  in  Chapter  11. 

4.  Recently,  J.  F.  Dalzell  has  engaged  in  significant  efforts  on  polyspectral  analysis. 
He  tried  not  only  bispectrum  but  even  trispectrum  analysis  in  the  study  of  seakeeping  in 
irregular  waves,  with  excellent  results,  although  an  enoimous  amount  of  careful  computa¬ 
tion  was  necessary  which  at  this  stage  might  not  be  feasible  for  practical  purposes. 

5.  Probability  distribution  of  the  maxima  and  the  minima  of  the  oscillatory  motions 
can  also  be  obtained  from  the  nonlinear  frequency  responses.  The  general  procedure  for 
obtaining  the  probability  distribution  of  extremes  by  calculating  the  cumulants  related  to 
the  nonlinear  frequency  response  characters  and  a  few  other  examples  are  reviewed  in 
Chapter  12. 
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6.  If  the  process  can  be  inverted  into  a  vector  Markov  process  by  state-space  irans- 
fonnations,  sometimes  the  solution  of  Fokker-Planck  equations  can  be  used  to  estimate 
the  joint  probability  distribution  function,  from  which  many  probabihty  characteristics  of 
the  origi^  process  can  be  derived,  even  for  nonlinear  oscillations. 

7.  Recently,  a  few  applications  of  this  approach  have  appeared  in  seakeeping 
studies.  However,  the  Fokker-Planck  equations  have  been  solved  only  for  some  limited 
cases,  and  are  not  so  familiar  to  engineers.  More  studies  are  necessary  for  naval  architects 
to  become  familiar  to  the  application  of  this  method. 

8.  Some  efforts  have  been  made  to  expand  the  model  fitting  method  to  the  nonlin¬ 
ear  process.  The  threshold  autoregressive  model,  the  exponential  autoregressive  model, 
and  the  nonlineai  threshold  model  are  examples  of  expanded  models.  Some  of  these  mod¬ 
els  loox  promising  and  some  are  under  active  development,  but  we  will  have  to  wait  until 
they  are  more  fully  formulated  to  accumulate  experience  in  apphcation  to  practicai 
problems. 

As  was  mentioned  in  the  Foreword,  the  contents  of  this  report  were  summarized  at 
the  time  of  this  author’s  oral  presentation  at  DTRC  in  July  1985,  reflecting  the  state  of  the 
art  up  to  1984,  though  the  written  version  was  completed  in  August  1987.  After  these 
dates,  the  state  of  the  art  has  made  considerable  progress,  especially  in  the  field  treated 
here  in  Part  m,  and  this  author  finds  the  ‘review,’  and  this  conclusion,  to  be  insufficient 
because  of  the  recent  works  of  several  researchers.  In  order  to  update  the  report,  this 
author  added  a  Supplement  of  References  101  through  124,  listing  pubhcahons  that  have 
appeared  since  1984,  together  with  some  other  publications  that  were  not  referred  to  in 
the  original  manuscript. 
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CHAPTER  15 


CONCLUSIONS  FOR  PART  I  THROUGH  PART  HI 

Part  I  of  this  lecture  note  summarized  the  conventional  procedures  for  analyzing  the 
irregular  time  histories  of  observed  data,  like  irregular  ocean  waves,  and  the  responses  of 
marine  vehicles  and  structures  on  the  ocean,  that  can  be  treated  as  weakly  stationary  er- 
godic  stochastic  processes.  In  these  analyses,  the  so-called  Wiener’s  general  harmonic 
analysis  technique  plays  a  large  role,  and  the  correlation  and  spectrum  functions  of  the 
processes  were  important  and  included  much  information  on  the  characteristics  of  the 
processes.  When  the  responses  of  some  dynamic  systems  to  random  excitation  are  to  be 
treated,  the  cross  relations  of  the  input  (excitation)  and  the  output  (response),  i.e.,  the 
cross  correlations  and  cross  spectra,  are  very  important. 

The  theories  of  the  analysis  are  complete  and  rather  beautiful.  However,  in  sample 
computations  from  practical  data,  that  is,  from  discrete  data  sampled  at  some  time  inter¬ 
val  of  finite  length,  many  statistical  considerations  are  necessary  in  estimating  the 
correlation  and  spectrum  functions  to  get  statistically  reUable  results.  After  the  general 
procedures  were  summarized,  the  discussion  concentrated  on  that  point.  In  these  proce¬ 
dures,  this  author  made  several  suggestions  for  improving  the  reliability  (acmally  the 
coherencies)  and  these  results  were  summarized.  This  author  stressed  that  we  should  pay 
more  attention  to  the  time-domain  characteristics  of  the  functions.  For  example,  the  cor- 
relogram,  that  is  the  diagram  of  the  correlation  function  and  is  a  function  in  the  time 
domain,  deserves  as  much  attention  as  is  now  paid  to  the  spectrum. 

Part  n  is  concerned  with  the  parametric  analysis  of  a  stationary  process,  which  is 
really  an  alternative  or  more  modem  method  for  analyzing  the  sample  random  process. 
This  author  beheves  that  the  characteristics  of  the  function  in  the  time  domain  play  bigger 
roles  in  this  method  than  in  the  nonparametric  conventional  method,  discussed  in  Part  I. 

The  parametric  method  rits  a  statistical  model  to  an  observed  process  and  estimates 
the  parameters  from  the  observed  data.  In  Pan  n,  the  autoregressive  (AR)  models,  mov¬ 
ing  average  (MA)  models,  and  mixed  autoregressive  moving  average  (ARMA)  models 
were  introduced.  Since  these  models  and  the  parametric  approach  are  not  familiar  to  most 
engineers,  especially  in  the  field  of  naval  architecture,  the  author  explained  them  in  con¬ 
siderable  detail.  (He  is  afraid  it  was  in  too  much  detail.) 

Usually  a  moderate  or  rather  low  order  finite  AR(n),  MA(m),  or  ARMA(n,m)  model 
can  be  fitted  to  represent  adequately  most  of  the  observed  processes.  The  optimum  order 
of  these  fitted  models,  n  or  m,  can  be  estimated  by  a  method  called  Akaike’s  information 
criteria  (AIC).  This  method  is  based  on  infonnation  theory  and  enables  one  to  choose  the 
order  n  or  m  that  maximizes  the  statistical  entropy  of  the  estimate.  It  is  therefore  called 
the  “Maximum  Entropy  Method,”  although  it  is  different  from  a  similar  method  already 
published  under  the  same  name.  The  AIC  method  can  give  the  optimum  order  to  be 
adopted.  In  Part  II,  in  explaining  the  characteristics  of  these  models,  this  author  generated 
several  fundamental  models,  the  AR(0)  (pure  random),  AR(1),  AR(2),  ARMA(2.1), 
MA(2),  MA(1),  and  ARMA(2.2)  models,  by  simulation  and  analyzed  them  by  the  para¬ 
metric  method.  The  orders  were  estimated  by  AIC  and,  for  almost  all  processes,  we 
succeeded  to  fit  models  that  coincided  with  the  models  used  to  generate  the  simulated 
process.  In  these  demonstrations,  the  simulated  processes  were  also  analyzed  by  the 
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nonparametric  cooventional  method  (sometimes  called  the  correlation  method,  or  the 
Blackman-Tukey,  BT  method),  and  the  spectra  were  compared  with  the  ones  obtained  by 
the  parametric  method.  From  these  comparisons,  we  can  get  an  idea  of  the  characteristics 
of  the  parametric  method.  Although  they  could  have  very  steep  peaks,  the  spectra  from 
the  parametric  method  are  smooth,  and  are  free  from  the  erroneous  smoothing  (blurring) 
effect  of  spectra]  windows  in  the  nonparametric  method.  They  are  differently  effected  by 
the  statistical  fluctuations  of  sample  estimation  that  come  from  the  flniteness  of  discrete 
data. 

Examples  of  results  obtained  by  applying  this  paramenic  method  to  the  analysis  of 
seakeeping  data  for  marine  vehicles  and  structures  were  also  given  to  demonstrate  the 
usefulness  of  this  approach  in  this  field. 

This  author  concludes  that  although  this  parametric  approach  is  not  the  ultimate 
method  of  course,  it  is  very  promising,  gives  reliable  results,  and  offers  a  supplement  to 
the  nonparametric  method  explained  in  Part  I.  The  nonparametric  method  is,  in  a  sense,  a 
method  for  estimating  infinite  or  very  large  number  of  parameters  from  finite  data,  or  for 
estimating  their  spectrum  or  correlation  functions.  The  parametric  method  on  the  other 
hand  is  a  method  for  estimating  a  finite  number  of  parameters  for  the  same  purpose. 

The  characteristics  of  a  dynamic  system,  which  is  usually  assumed  to  be  determinis¬ 
tic,  must  be  approximated  by  a  fmite  number  of  parameters  because  the  system  is  usually 
governed  by  equations  of  motions  with  a  certain  finite  ntimber  of  parameters.  The  types 
and  also  the  order  of  parametric  models  are  closely  related  with  the  order  of  the  equations 
of  motions,  so  the  parametric  method  can  be  presumed  to  be  more  reasonable  in  the  anal¬ 
ysis  of  response  characteristics. 

In  Parts  I  and  H,  the  processes  were  assumed  to  be  linear.  In  Part  HI,  the  method  for 
treating  a  nonlinear  process  was  summarized  and  several  methods,  such  as  the  lineariza¬ 
tion  method,  the  perturbation  method,  and  the  Voltera  expansion  method  (polyspectra 
method),  were  reviewed  and  the  relations  of  these  methods  to  each  other  were  described. 
Then,  in  relation  to  the  probability  distribution  of  the  extreme  amplitudes  of  nonlinear 
processes,  the  application  of  the  Fokker-Planck  equation  and  some  examples  appearing 
recently  in  naval  architecture  were  introduced.  The  probability  distribution  of  extremes  of 
nonlinear  response  process  were  also  related  to  the  Ugher  order  frequency  response  func¬ 
tions,  and  can  be  connected  through  the  cumulant  expressions.  The  general  relations  of 
these  functions  were  summarized.  The  achievements  of  J.  F.  Dalzell  on  the  application  of 
poiyspectra  and  in  the  derivation  of  the  probability  distributions  of  extremes  of  seakeep- 
ing  data  were  introduced  at  length.  At  the  end  of  Part  m,  as  an  extension  of  the  model 
fitting  approach  treated  in  Part  n,  a  few  trials  on  nonlinear  parametric  models  proposed 
by  several  statisticians  were  introduced.  Some  of  these  trials  look  promising  and  are  still 
under  active  development.  However,  as  has  been  concluded  in  (Chapter  14,  we  need  more 
experience  with  such  applications. 
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